2024-03-26T17:39:35.619 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-26T17:39:35.619 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/postgres}} --owner scheduled_lflores@teuthology --name lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi -- /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762/orig.config.yaml 2024-03-26T17:39:35.685 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T17:39:35.794 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762 branch: wip-batrick-testing-20240320.145742 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/postgres}} email: null first_in_suite: false job_id: '7623762' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240320.145742 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - \(PG_DEGRADED - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 360516069d9393362c4cc6eb9371680fe16d66ab subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 360516069d9393362c4cc6eb9371680fe16d66ab kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-tracker-65018 sha1: 72e2437271dcafb9826f2f49bb1001426fa6de81 owner: scheduled_lflores@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4067 sha1: 360516069d9393362c4cc6eb9371680fe16d66ab sleep_before_teardown: 0 subset: 502/512 suite: fs suite_branch: wip-tracker-65018 suite_path: /home/teuthworker/src/github.com_ljflores_ceph_72e2437271dcafb9826f2f49bb1001426fa6de81/qa suite_relpath: qa suite_repo: https://github.com/ljflores/ceph.git suite_sha1: 72e2437271dcafb9826f2f49bb1001426fa6de81 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKSwWzaVsQgro5CHYdbCGUCyWtTXpzEb4NBu7ezmRUIidELmbvYNSdez4ZexFL60G2cSIOOcpRf8NeeQ9TXvSw= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6KAuB0jh54iZe4a5iZSyUd3xUe3stTD7sw9okSnwMcaeWShUX0nxsPHLCM+ZLhRp3JcOQj8I6QzwvxKMlbI2U= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLr9QCKMWUvmLDACiovXBf53HiWiFU500HXopFgmLU+nelvqWgui3p7LPK/xfs1uefYDg0A95ebaoqhCP7SyEg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: - /cephfs/home/lflores/src/github.com_ljflores_ceph_wip-tracker-65018/qa/suites/fs/workload/overrides/pg_health.yaml meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-26_17:10:07 tube: smithi user: lflores verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-26T17:39:35.794 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ljflores_ceph_72e2437271dcafb9826f2f49bb1001426fa6de81/qa; will attempt to use it 2024-03-26T17:39:35.795 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ljflores_ceph_72e2437271dcafb9826f2f49bb1001426fa6de81/qa/tasks 2024-03-26T17:39:35.795 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-26T17:39:35.797 INFO:teuthology.task.internal:Checking packages... 2024-03-26T17:39:35.820 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '360516069d9393362c4cc6eb9371680fe16d66ab' 2024-03-26T17:39:35.820 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-26T17:39:35.820 INFO:teuthology.packaging:ref: None 2024-03-26T17:39:35.820 INFO:teuthology.packaging:tag: None 2024-03-26T17:39:35.820 INFO:teuthology.packaging:branch: wip-batrick-testing-20240320.145742 2024-03-26T17:39:35.820 INFO:teuthology.packaging:sha1: 360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:39:35.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240320.145742 2024-03-26T17:39:36.106 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2311.g36051606 2024-03-26T17:39:36.107 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-26T17:39:36.114 INFO:teuthology.task.internal:no buildpackages task found 2024-03-26T17:39:36.114 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-26T17:39:36.126 INFO:teuthology.task.internal:Saving configuration 2024-03-26T17:39:36.151 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-26T17:39:36.157 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T17:39:36.176 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:33:06.801470', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKSwWzaVsQgro5CHYdbCGUCyWtTXpzEb4NBu7ezmRUIidELmbvYNSdez4ZexFL60G2cSIOOcpRf8NeeQ9TXvSw='} 2024-03-26T17:39:36.195 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:33:06.799714', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6KAuB0jh54iZe4a5iZSyUd3xUe3stTD7sw9okSnwMcaeWShUX0nxsPHLCM+ZLhRp3JcOQj8I6QzwvxKMlbI2U='} 2024-03-26T17:39:36.218 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:33:06.802503', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLr9QCKMWUvmLDACiovXBf53HiWiFU500HXopFgmLU+nelvqWgui3p7LPK/xfs1uefYDg0A95ebaoqhCP7SyEg='} 2024-03-26T17:39:36.218 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-26T17:39:36.224 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-26T17:39:36.224 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-26T17:39:36.225 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-26T17:39:36.225 INFO:teuthology.run_tasks:Running task console_log... 2024-03-26T17:39:36.359 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbad3f12160>, signals=[15]) 2024-03-26T17:39:36.359 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-26T17:39:36.417 INFO:teuthology.task.internal:Opening connections... 2024-03-26T17:39:36.417 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T17:39:36.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:39:36.524 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-03-26T17:39:36.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:39:36.615 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-03-26T17:39:36.616 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:39:36.706 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-26T17:39:36.713 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-03-26T17:39:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-03-26T17:39:36.732 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-03-26T17:39:36.789 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-03-26T17:39:36.790 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T17:39:36.791 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T17:39:36.791 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T17:39:36.791 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-03-26T17:39:36.817 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-03-26T17:39:36.835 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-03-26T17:39:36.835 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-03-26T17:39:36.892 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-03-26T17:39:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T17:39:36.894 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T17:39:36.894 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T17:39:36.894 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-03-26T17:39:36.917 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-03-26T17:39:36.934 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-03-26T17:39:36.934 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-03-26T17:39:36.992 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-03-26T17:39:36.992 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-03-26T17:39:36.992 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-03-26T17:39:36.992 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-03-26T17:39:36.992 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-03-26T17:39:36.992 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-03-26T17:39:36.992 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T17:39:36.993 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-03-26T17:39:36.993 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-03-26T17:39:36.993 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T17:39:36.993 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-03-26T17:39:36.993 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T17:39:36.993 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T17:39:36.993 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T17:39:36.993 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-03-26T17:39:37.017 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-26T17:39:37.027 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-26T17:39:37.081 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-26T17:39:37.081 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-03-26T17:39:37.084 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-03-26T17:39:37.086 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-03-26T17:39:37.103 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-26T17:39:37.122 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-26T17:39:37.123 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-03-26T17:39:37.141 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-03-26T17:39:37.144 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-03-26T17:39:37.174 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-26T17:39:37.251 INFO:teuthology.run_tasks:Running task kernel... 2024-03-26T17:39:37.265 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-26T17:39:37.265 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-26T17:39:37.266 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-26T17:39:37.266 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T17:39:37.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:39:37.283 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-03-26T17:39:37.341 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-361.el9.x86_64 2024-03-26T17:39:37.341 INFO:teuthology.task.kernel:Running kernel on smithi049: 5.14.0-361.el9.x86_64 2024-03-26T17:39:37.341 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum install -y kernel 2024-03-26T17:39:39.848 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 7.9 MB/s | 6.7 MB 00:00 2024-03-26T17:39:41.473 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.0 MB 00:00 2024-03-26T17:39:43.089 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 8.0 MB 00:01 2024-03-26T17:39:46.490 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-03-26T17:39:50.941 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.1 MB 00:00 2024-03-26T17:39:52.516 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-03-26T17:39:54.537 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-26T17:39:59.651 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-03-26T17:40:01.816 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-26T17:40:01.816 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-26T17:40:01.906 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-03-26T17:40:01.915 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-03-26T17:40:01.915 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-03-26T17:40:01.915 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-03-26T17:40:01.915 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-03-26T17:40:01.915 INFO:teuthology.orchestra.run.smithi049.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-26T17:40:01.915 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-03-26T17:40:01.915 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-26T17:40:01.915 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-26T17:40:01.915 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-26T17:40:01.915 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:40:01.916 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-03-26T17:40:01.916 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-03-26T17:40:01.916 INFO:teuthology.orchestra.run.smithi049.stdout:Install 4 Packages 2024-03-26T17:40:01.916 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:40:01.917 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 76 M 2024-03-26T17:40:01.917 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 123 M 2024-03-26T17:40:01.917 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-03-26T17:40:02.211 INFO:teuthology.orchestra.run.smithi049.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 333 kB/s | 66 kB 00:00 2024-03-26T17:40:04.585 INFO:teuthology.orchestra.run.smithi049.stdout:(2/4): kernel-core-5.14.0-432.el9.x86_64.rpm 5.8 MB/s | 15 MB 00:02 2024-03-26T17:40:05.277 INFO:teuthology.orchestra.run.smithi049.stdout:(3/4): kernel-modules-core-5.14.0-432.el9.x86_6 9.0 MB/s | 27 MB 00:03 2024-03-26T17:40:05.586 INFO:teuthology.orchestra.run.smithi049.stdout:(4/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 9.4 MB/s | 33 MB 00:03 2024-03-26T17:40:05.586 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:40:05.587 INFO:teuthology.orchestra.run.smithi049.stdout:Total 21 MB/s | 76 MB 00:03 2024-03-26T17:40:06.295 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-03-26T17:40:06.736 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-03-26T17:40:06.736 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-03-26T17:40:07.802 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-03-26T17:40:07.802 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-03-26T17:40:10.404 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-03-26T17:40:11.021 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-26T17:40:11.103 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:40:13.961 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:40:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:40:19.169 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:40:19.215 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:40:24.318 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:40:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:40:55.037 INFO:teuthology.orchestra.run.smithi049.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T17:40:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:40:55.082 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:40:55.494 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:40:55.494 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-26T17:40:55.494 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:40:55.494 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:40:55.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:40:55.921 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:40:55.921 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-03-26T17:40:55.921 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-26T17:40:55.921 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-26T17:40:55.921 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-26T17:40:55.921 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-26T17:40:55.922 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:40:55.922 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-03-26T17:40:56.245 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T17:40:56.350 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T17:40:56.351 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T17:40:56.351 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-26T17:40:56.351 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-26T17:40:56.351 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-26T17:40:56.351 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-26T17:40:56.351 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=None, version=distro) 2024-03-26T17:40:56.352 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T17:40:56.434 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T17:40:56.435 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T17:40:56.435 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -qi grub2-tools 2024-03-26T17:40:56.474 INFO:teuthology.orchestra.run.smithi049.stdout:Name : grub2-tools 2024-03-26T17:40:56.474 INFO:teuthology.orchestra.run.smithi049.stdout:Epoch : 1 2024-03-26T17:40:56.474 INFO:teuthology.orchestra.run.smithi049.stdout:Version : 2.06 2024-03-26T17:40:56.474 INFO:teuthology.orchestra.run.smithi049.stdout:Release : 68.el9 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: x86_64 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Group : Unspecified 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Size : 8264254 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:License : GPLv3+ 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Packager : builder@centos.org 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Vendor : CentOS 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T17:40:56.475 INFO:teuthology.orchestra.run.smithi049.stdout:Summary : Support tools for GRUB. 2024-03-26T17:40:56.476 INFO:teuthology.orchestra.run.smithi049.stdout:Description : 2024-03-26T17:40:56.476 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:40:56.476 INFO:teuthology.orchestra.run.smithi049.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T17:40:56.476 INFO:teuthology.orchestra.run.smithi049.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T17:40:56.476 INFO:teuthology.orchestra.run.smithi049.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T17:40:56.476 INFO:teuthology.orchestra.run.smithi049.stdout:hardware devices. 2024-03-26T17:40:56.476 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:40:56.476 INFO:teuthology.orchestra.run.smithi049.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T17:40:56.477 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T17:40:56.477 INFO:teuthology.task.kernel:Updating grub on smithi049 to boot 5.14.0-432.el9.x86_64 2024-03-26T17:40:56.477 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T17:40:57.086 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2024-03-26T17:40:58.507 INFO:teuthology.orchestra.run.smithi049.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T17:40:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:done 2024-03-26T17:40:58.536 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-03-26T17:40:58.552 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.8IWXUToaNu 2024-03-26T17:40:58.552 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8IWXUToaNu 2024-03-26T17:40:58.628 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.8IWXUToaNu 2024-03-26T17:40:58.771 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.8IWXUToaNu is 6KB 2024-03-26T17:40:58.792 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.8IWXUToaNu 2024-03-26T17:40:58.806 DEBUG:teuthology.orchestra.run.smithi049:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T17:40:58.876 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T17:40:58.876 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T17:40:58.876 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T17:40:58.876 INFO:teuthology.orchestra.run.smithi049.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-26T17:40:58.877 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-26T17:40:59.032 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2024-03-26T17:41:29.034 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-26T17:41:29.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-26T17:41:29.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:42:03.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-26T17:42:12.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-26T17:42:12.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:42:15.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-26T17:42:27.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-26T17:42:27.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:42:30.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-26T17:42:45.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-26T17:42:45.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:42:48.770 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-03-26T17:43:06.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-26T17:43:06.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:43:38.563 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-03-26T17:43:38.984 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-26T17:43:38.984 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-432.el9.x86_64"... 2024-03-26T17:43:38.984 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-03-26T17:43:39.000 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-432.el9.x86_64 2024-03-26T17:43:39.001 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-26T17:43:39.001 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T17:43:39.001 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T17:43:40.002 DEBUG:teuthology.parallel:result is None 2024-03-26T17:43:40.003 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-26T17:43:40.013 INFO:teuthology.task.internal:Creating test directory... 2024-03-26T17:43:40.013 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T17:43:40.016 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T17:43:40.018 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T17:43:40.036 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-26T17:43:40.042 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-26T17:43:40.052 INFO:teuthology.task.internal:Creating archive directory... 2024-03-26T17:43:40.053 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T17:43:40.074 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T17:43:40.076 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T17:43:40.202 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-26T17:43:40.209 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-26T17:43:40.210 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T17:43:40.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:43:40.226 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T17:43:40.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:43:40.243 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T17:43:40.246 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T17:43:40.270 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T17:43:40.304 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:43:40.314 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:43:40.337 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:43:40.339 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:43:40.361 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:43:40.363 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:43:40.365 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-26T17:43:40.373 INFO:teuthology.task.internal:Configuring sudo... 2024-03-26T17:43:40.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T17:43:40.375 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T17:43:40.378 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T17:43:40.408 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-26T17:43:40.418 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-26T17:43:40.418 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T17:43:40.450 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T17:43:40.453 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T17:43:40.469 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:43:40.541 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:43:40.603 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:43:40.686 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:43:40.735 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:43:40.736 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T17:43:40.808 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:43:40.852 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:43:40.904 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:43:40.989 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:43:41.036 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:43:41.036 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T17:43:41.106 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:43:41.149 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:43:41.210 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:43:41.295 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:43:41.339 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:43:41.339 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T17:43:41.408 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-03-26T17:43:41.410 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-03-26T17:43:41.413 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-03-26T17:43:41.454 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T17:43:41.463 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T17:43:41.483 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T17:43:41.980 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-26T17:43:41.989 INFO:teuthology.task.internal:Starting timer... 2024-03-26T17:43:41.989 INFO:teuthology.run_tasks:Running task pcp... 2024-03-26T17:43:42.005 INFO:teuthology.run_tasks:Running task selinux... 2024-03-26T17:43:42.014 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-26T17:43:42.015 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-03-26T17:43:42.087 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-03-26T17:43:42.088 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-03-26T17:43:42.137 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-03-26T17:43:42.138 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-03-26T17:43:42.193 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-03-26T17:43:42.194 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-26T17:43:42.194 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-03-26T17:43:42.220 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-03-26T17:43:42.221 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-03-26T17:43:42.245 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-03-26T17:43:42.246 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-03-26T17:43:42.270 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-03-26T17:43:42.271 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-03-26T17:43:42.271 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T17:43:42.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:43:42.306 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T17:43:42.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:43:42.335 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T17:43:42.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:43:42.365 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-26T17:43:42.365 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-03-26T17:43:42.406 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-03-26T17:43:42.456 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-03-26T17:43:42.487 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-26T17:43:42.496 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-26T17:43:42.502 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-26T17:43:42.627 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-26T17:43:42.643 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-26T17:43:42.645 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi115.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-26T17:48:32.499 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-03-26T17:48:32.500 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-26T17:48:32.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:48:32.609 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-03-26T17:48:32.694 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-03-26T17:48:32.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-03-26T17:48:32.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:48:32.786 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-03-26T17:48:32.857 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-03-26T17:48:32.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-03-26T17:48:32.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:48:32.948 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-03-26T17:48:32.973 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-03-26T17:48:32.974 INFO:teuthology.run_tasks:Running task clock... 2024-03-26T17:48:32.985 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-26T17:48:32.985 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T17:48:32.985 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T17:48:32.988 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T17:48:32.988 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T17:48:32.990 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T17:48:32.990 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T17:48:33.020 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T17:48:33.023 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T17:48:33.037 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T17:48:33.044 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T17:48:33.045 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T17:48:33.062 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T17:48:33.066 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-03-26T17:48:33.080 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-03-26T17:48:33.081 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-03-26T17:48:33.090 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-03-26T17:48:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T17:48:33.098 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-03-26T17:48:33.104 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-03-26T17:48:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T17:48:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T17:48:33.121 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T17:48:33.137 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T17:48:33.138 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T17:48:33.156 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-03-26T17:48:33.160 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T17:48:33.160 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-03-26T17:48:33.160 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.160 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.160 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.160 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.181 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-03-26T17:48:33.184 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T17:48:33.184 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-03-26T17:48:33.184 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.184 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.184 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.184 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.191 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-03-26T17:48:33.194 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T17:48:33.195 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-03-26T17:48:33.195 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.195 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.195 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.195 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:48:33.196 INFO:teuthology.run_tasks:Running task install... 2024-03-26T17:48:33.205 DEBUG:teuthology.task.install:project ceph 2024-03-26T17:48:33.205 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab'}} 2024-03-26T17:48:33.205 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab'} 2024-03-26T17:48:33.205 INFO:teuthology.task.install:Using flavor: default 2024-03-26T17:48:33.211 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-26T17:48:33.211 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-26T17:48:33.211 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'tag': None, 'wait_for_package': False} 2024-03-26T17:48:33.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:48:33.214 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'tag': None, 'wait_for_package': False} 2024-03-26T17:48:33.214 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:48:33.215 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'tag': None, 'wait_for_package': False} 2024-03-26T17:48:33.215 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:48:33.370 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/ 2024-03-26T17:48:33.370 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2311.g36051606 2024-03-26T17:48:33.460 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/ 2024-03-26T17:48:33.460 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2311.g36051606 2024-03-26T17:48:33.464 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/ 2024-03-26T17:48:33.464 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2311.g36051606 2024-03-26T17:48:33.480 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T17:48:33.480 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:48:33.481 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T17:48:33.516 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T17:48:33.517 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/360516069d9393362c4cc6eb9371680fe16d66ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T17:48:33.590 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T17:48:33.660 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T17:48:33.660 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:48:33.660 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T17:48:33.672 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T17:48:33.672 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:48:33.672 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T17:48:33.677 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T17:48:33.689 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T17:48:33.689 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/360516069d9393362c4cc6eb9371680fe16d66ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T17:48:33.700 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T17:48:33.700 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/360516069d9393362c4cc6eb9371680fe16d66ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T17:48:33.706 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-03-26T17:48:33.708 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-03-26T17:48:33.757 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T17:48:33.768 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T17:48:33.837 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T17:48:33.847 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T17:48:33.904 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-03-26T17:48:33.905 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-03-26T17:48:33.913 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-03-26T17:48:33.915 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-03-26T17:48:33.964 INFO:teuthology.orchestra.run.smithi049.stdout:74 files removed 2024-03-26T17:48:33.991 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T17:48:34.140 INFO:teuthology.orchestra.run.smithi171.stdout:74 files removed 2024-03-26T17:48:34.148 INFO:teuthology.orchestra.run.smithi115.stdout:74 files removed 2024-03-26T17:48:34.167 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T17:48:34.175 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T17:48:34.701 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 214 kB/s | 83 kB 00:00 2024-03-26T17:48:34.799 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 247 kB/s | 83 kB 00:00 2024-03-26T17:48:34.869 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-03-26T17:48:35.075 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-03-26T17:48:35.269 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 28 kB/s | 10 kB 00:00 2024-03-26T17:48:35.433 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 17 kB/s | 10 kB 00:00 2024-03-26T17:48:35.450 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-03-26T17:48:35.620 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-03-26T17:48:35.775 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-03-26T17:48:36.047 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-26T17:48:36.294 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-26T17:48:36.303 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - AppStream 8.2 MB/s | 6.7 MB 00:00 2024-03-26T17:48:37.456 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-03-26T17:48:37.686 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-26T17:48:37.845 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-03-26T17:48:38.826 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.0 MB 00:00 2024-03-26T17:48:39.622 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.0 MB 00:01 2024-03-26T17:48:41.160 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-03-26T17:48:45.745 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.1 MB 00:01 2024-03-26T17:48:47.310 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 903 kB/s | 8.0 MB 00:09 2024-03-26T17:48:47.437 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-03-26T17:48:49.397 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-26T17:48:54.246 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-03-26T17:48:56.671 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:48:56.672 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:48:56.684 INFO:teuthology.orchestra.run.smithi115.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T17:48:56.685 INFO:teuthology.orchestra.run.smithi115.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T17:48:56.686 INFO:teuthology.orchestra.run.smithi115.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T17:48:56.687 INFO:teuthology.orchestra.run.smithi115.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T17:48:56.689 INFO:teuthology.orchestra.run.smithi115.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T17:48:56.691 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T17:48:56.692 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T17:48:56.771 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-03-26T17:48:56.788 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================ 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================ 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:19.0.0-2311.g36051606.el9 ceph 6.6 k 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.4 M 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 890 k 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2311.g36051606.el9 ceph 145 k 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:19.0.0-2311.g36051606.el9 ceph 1.5 M 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 140 k 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 1.8 M 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 7.4 M 2024-03-26T17:48:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 49 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 9.8 M 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:19.0.0-2311.g36051606.el9 ceph 48 M 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 264 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 750 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 221 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell x86_64 2:19.0.0-2311.g36051606.el9 ceph 27 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 18 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 32 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 746 k 2024-03-26T17:48:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 127 k 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:19.0.0-2311.g36051606.el9 ceph 161 k 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:19.0.0-2311.g36051606.el9 ceph 321 k 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 298 k 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 99 k 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 85 k 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:48:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 170 k 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T17:48:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.4 M 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T17:48:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 21 M 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 34 k 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:19.0.0-2311.g36051606.el9 ceph 2.4 M 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 248 k 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:19.0.0-2311.g36051606.el9 ceph 4.7 M 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:19.0.0-2311.g36051606.el9 ceph 17 M 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 16 k 2024-03-26T17:48:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:19.0.0-2311.g36051606.el9 ceph 25 k 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:19.0.0-2311.g36051606.el9 ceph 163 k 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T17:48:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 500 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.1 M 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T17:48:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T17:48:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:19.0.0-2311.g36051606.el9 ceph 45 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 131 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T17:48:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T17:48:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T17:48:56.800 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T17:48:56.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-03-26T17:48:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T17:48:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:48:56.803 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-03-26T17:48:56.803 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================ 2024-03-26T17:48:56.803 INFO:teuthology.orchestra.run.smithi115.stdout:Install 150 Packages 2024-03-26T17:48:56.803 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 24 Packages 2024-03-26T17:48:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:48:56.803 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 225 M 2024-03-26T17:48:56.803 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-03-26T17:48:57.821 INFO:teuthology.orchestra.run.smithi115.stdout:(1/174): ceph-19.0.0-2311.g36051606.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-03-26T17:48:58.121 INFO:teuthology.orchestra.run.smithi115.stdout:(2/174): ceph-base-19.0.0-2311.g36051606.el9.x8 9.4 MB/s | 5.4 MB 00:00 2024-03-26T17:48:58.196 INFO:teuthology.orchestra.run.smithi115.stdout:(3/174): ceph-immutable-object-cache-19.0.0-231 1.9 MB/s | 145 kB 00:00 2024-03-26T17:48:58.447 INFO:teuthology.orchestra.run.smithi115.stdout:(4/174): ceph-mds-19.0.0-2311.g36051606.el9.x86 9.4 MB/s | 2.4 MB 00:00 2024-03-26T17:48:58.622 INFO:teuthology.orchestra.run.smithi115.stdout:(5/174): ceph-mgr-19.0.0-2311.g36051606.el9.x86 8.5 MB/s | 1.5 MB 00:00 2024-03-26T17:48:58.647 INFO:teuthology.orchestra.run.smithi115.stdout:(6/174): ceph-fuse-19.0.0-2311.g36051606.el9.x8 1.1 MB/s | 890 kB 00:00 2024-03-26T17:48:59.014 INFO:teuthology.orchestra.run.smithi115.stdout:(7/174): ceph-mon-19.0.0-2311.g36051606.el9.x86 12 MB/s | 4.7 MB 00:00 2024-03-26T17:48:59.281 INFO:teuthology.orchestra.run.smithi115.stdout:(8/174): ceph-common-19.0.0-2311.g36051606.el9. 12 MB/s | 21 MB 00:01 2024-03-26T17:48:59.331 INFO:teuthology.orchestra.run.smithi115.stdout:(9/174): ceph-selinux-19.0.0-2311.g36051606.el9 505 kB/s | 25 kB 00:00 2024-03-26T17:48:59.418 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 19 MB 00:11 2024-03-26T17:48:59.690 INFO:teuthology.orchestra.run.smithi115.stdout:(10/174): ceph-radosgw-19.0.0-2311.g36051606.el 15 MB/s | 9.8 MB 00:00 2024-03-26T17:48:59.790 INFO:teuthology.orchestra.run.smithi115.stdout:(11/174): cephfs-mirror-19.0.0-2311.g36051606.e 2.2 MB/s | 221 kB 00:00 2024-03-26T17:48:59.857 INFO:teuthology.orchestra.run.smithi115.stdout:(12/174): cephfs-shell-19.0.0-2311.g36051606.el 407 kB/s | 27 kB 00:00 2024-03-26T17:48:59.924 INFO:teuthology.orchestra.run.smithi115.stdout:(13/174): libcephfs-devel-19.0.0-2311.g36051606 475 kB/s | 32 kB 00:00 2024-03-26T17:49:00.016 INFO:teuthology.orchestra.run.smithi115.stdout:(14/174): libcephfs2-19.0.0-2311.g36051606.el9. 8.0 MB/s | 746 kB 00:00 2024-03-26T17:49:00.091 INFO:teuthology.orchestra.run.smithi115.stdout:(15/174): libcephsqlite-19.0.0-2311.g36051606.e 2.1 MB/s | 163 kB 00:00 2024-03-26T17:49:00.175 INFO:teuthology.orchestra.run.smithi115.stdout:(16/174): librados-devel-19.0.0-2311.g36051606. 1.5 MB/s | 127 kB 00:00 2024-03-26T17:49:00.267 INFO:teuthology.orchestra.run.smithi115.stdout:(17/174): libradosstriper1-19.0.0-2311.g3605160 5.3 MB/s | 500 kB 00:00 2024-03-26T17:49:00.742 INFO:teuthology.orchestra.run.smithi115.stdout:(18/174): librgw2-19.0.0-2311.g36051606.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-26T17:49:00.784 INFO:teuthology.orchestra.run.smithi115.stdout:(19/174): python3-ceph-argparse-19.0.0-2311.g36 1.1 MB/s | 45 kB 00:00 2024-03-26T17:49:00.834 INFO:teuthology.orchestra.run.smithi115.stdout:(20/174): python3-ceph-common-19.0.0-2311.g3605 2.6 MB/s | 131 kB 00:00 2024-03-26T17:49:00.926 INFO:teuthology.orchestra.run.smithi115.stdout:(21/174): python3-cephfs-19.0.0-2311.g36051606. 1.7 MB/s | 161 kB 00:00 2024-03-26T17:49:00.985 INFO:teuthology.orchestra.run.smithi115.stdout:(22/174): python3-rados-19.0.0-2311.g36051606.e 5.4 MB/s | 321 kB 00:00 2024-03-26T17:49:01.052 INFO:teuthology.orchestra.run.smithi115.stdout:(23/174): python3-rbd-19.0.0-2311.g36051606.el9 4.4 MB/s | 298 kB 00:00 2024-03-26T17:49:01.119 INFO:teuthology.orchestra.run.smithi115.stdout:(24/174): python3-rgw-19.0.0-2311.g36051606.el9 1.5 MB/s | 99 kB 00:00 2024-03-26T17:49:01.177 INFO:teuthology.orchestra.run.smithi115.stdout:(25/174): rbd-fuse-19.0.0-2311.g36051606.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-26T17:49:01.469 INFO:teuthology.orchestra.run.smithi115.stdout:(26/174): rbd-mirror-19.0.0-2311.g36051606.el9. 11 MB/s | 3.1 MB 00:00 2024-03-26T17:49:01.536 INFO:teuthology.orchestra.run.smithi115.stdout:(27/174): rbd-nbd-19.0.0-2311.g36051606.el9.x86 2.5 MB/s | 170 kB 00:00 2024-03-26T17:49:01.595 INFO:teuthology.orchestra.run.smithi115.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2311.g 584 kB/s | 34 kB 00:00 2024-03-26T17:49:01.662 INFO:teuthology.orchestra.run.smithi115.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2311.g3605160 2.1 MB/s | 140 kB 00:00 2024-03-26T17:49:01.862 INFO:teuthology.orchestra.run.smithi115.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2311.g36051 9.0 MB/s | 1.8 MB 00:00 2024-03-26T17:49:02.454 INFO:teuthology.orchestra.run.smithi115.stdout:(31/174): ceph-osd-19.0.0-2311.g36051606.el9.x8 4.4 MB/s | 17 MB 00:03 2024-03-26T17:49:02.554 INFO:teuthology.orchestra.run.smithi115.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-03-26T17:49:02.580 INFO:teuthology.orchestra.run.smithi115.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2311.g36 1.9 MB/s | 248 kB 00:00 2024-03-26T17:49:02.613 INFO:teuthology.orchestra.run.smithi115.stdout:(34/174): ceph-mgr-rook-19.0.0-2311.g36051606.e 848 kB/s | 49 kB 00:00 2024-03-26T17:49:02.646 INFO:teuthology.orchestra.run.smithi115.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2311.g3 246 kB/s | 16 kB 00:00 2024-03-26T17:49:02.672 INFO:teuthology.orchestra.run.smithi115.stdout:(36/174): ceph-volume-19.0.0-2311.g36051606.el9 4.4 MB/s | 264 kB 00:00 2024-03-26T17:49:03.106 INFO:teuthology.orchestra.run.smithi115.stdout:(37/174): ceph-test-19.0.0-2311.g36051606.el9.x 13 MB/s | 48 MB 00:03 2024-03-26T17:49:03.122 INFO:teuthology.orchestra.run.smithi115.stdout:(38/174): cephfs-top-19.0.0-2311.g36051606.el9. 40 kB/s | 18 kB 00:00 2024-03-26T17:49:03.156 INFO:teuthology.orchestra.run.smithi115.stdout:(39/174): cephadm-19.0.0-2311.g36051606.el9.noa 1.4 MB/s | 750 kB 00:00 2024-03-26T17:49:03.356 INFO:teuthology.orchestra.run.smithi115.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 714 kB/s | 106 kB 00:00 2024-03-26T17:49:03.382 INFO:teuthology.orchestra.run.smithi115.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 662 kB 00:00 2024-03-26T17:49:03.415 INFO:teuthology.orchestra.run.smithi115.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 920 kB 00:00 2024-03-26T17:49:03.440 INFO:teuthology.orchestra.run.smithi115.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-03-26T17:49:03.458 INFO:teuthology.orchestra.run.smithi115.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 419 kB/s | 32 kB 00:00 2024-03-26T17:49:03.591 INFO:teuthology.orchestra.run.smithi115.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-03-26T17:49:03.616 INFO:teuthology.orchestra.run.smithi115.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 64 kB/s | 10 kB 00:00 2024-03-26T17:49:03.641 INFO:teuthology.orchestra.run.smithi115.stdout:(47/174): flexiblas-openblas-openmp-3.0.4-8.el9 94 kB/s | 17 kB 00:00 2024-03-26T17:49:03.666 INFO:teuthology.orchestra.run.smithi115.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-26T17:49:03.692 INFO:teuthology.orchestra.run.smithi115.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 589 kB/s | 44 kB 00:00 2024-03-26T17:49:03.734 INFO:teuthology.orchestra.run.smithi115.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-26T17:49:03.759 INFO:teuthology.orchestra.run.smithi115.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.1 MB/s | 578 kB 00:00 2024-03-26T17:49:03.784 INFO:teuthology.orchestra.run.smithi115.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 511 kB/s | 46 kB 00:00 2024-03-26T17:49:03.809 INFO:teuthology.orchestra.run.smithi115.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-26T17:49:03.835 INFO:teuthology.orchestra.run.smithi115.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T17:49:03.860 INFO:teuthology.orchestra.run.smithi115.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-26T17:49:03.868 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.1 MB 00:00 2024-03-26T17:49:03.885 INFO:teuthology.orchestra.run.smithi115.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 458 kB/s | 34 kB 00:00 2024-03-26T17:49:03.936 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 831 kB/s | 19 MB 00:23 2024-03-26T17:49:03.952 INFO:teuthology.orchestra.run.smithi115.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-03-26T17:49:04.060 INFO:teuthology.orchestra.run.smithi115.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-03-26T17:49:04.086 INFO:teuthology.orchestra.run.smithi115.stdout:(59/174): python3-attrs-20.3.0-7.el9.noarch.rpm 390 kB/s | 78 kB 00:00 2024-03-26T17:49:04.178 INFO:teuthology.orchestra.run.smithi115.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-03-26T17:49:04.203 INFO:teuthology.orchestra.run.smithi115.stdout:(61/174): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-26T17:49:04.228 INFO:teuthology.orchestra.run.smithi115.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 130 kB/s | 18 kB 00:00 2024-03-26T17:49:04.253 INFO:teuthology.orchestra.run.smithi115.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-26T17:49:04.278 INFO:teuthology.orchestra.run.smithi115.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 636 kB/s | 32 kB 00:00 2024-03-26T17:49:04.407 INFO:teuthology.orchestra.run.smithi115.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-03-26T17:49:04.430 INFO:teuthology.orchestra.run.smithi115.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-03-26T17:49:04.455 INFO:teuthology.orchestra.run.smithi115.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-26T17:49:04.472 INFO:teuthology.orchestra.run.smithi115.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 69 kB 00:00 2024-03-26T17:49:04.489 INFO:teuthology.orchestra.run.smithi115.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2024-03-26T17:49:04.515 INFO:teuthology.orchestra.run.smithi115.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-26T17:49:04.540 INFO:teuthology.orchestra.run.smithi115.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-26T17:49:04.557 INFO:teuthology.orchestra.run.smithi115.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-26T17:49:04.582 INFO:teuthology.orchestra.run.smithi115.stdout:(73/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-26T17:49:04.599 INFO:teuthology.orchestra.run.smithi115.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 731 kB/s | 43 kB 00:00 2024-03-26T17:49:04.824 INFO:teuthology.orchestra.run.smithi115.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 173 kB/s | 39 kB 00:00 2024-03-26T17:49:04.841 INFO:teuthology.orchestra.run.smithi115.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 171 kB/s | 41 kB 00:00 2024-03-26T17:49:05.050 INFO:teuthology.orchestra.run.smithi115.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-03-26T17:49:05.075 INFO:teuthology.orchestra.run.smithi115.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2024-03-26T17:49:05.091 INFO:teuthology.orchestra.run.smithi115.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 227 kB/s | 56 kB 00:00 2024-03-26T17:49:05.108 INFO:teuthology.orchestra.run.smithi115.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-03-26T17:49:05.125 INFO:teuthology.orchestra.run.smithi115.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-26T17:49:05.142 INFO:teuthology.orchestra.run.smithi115.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-26T17:49:05.168 INFO:teuthology.orchestra.run.smithi115.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-26T17:49:05.201 INFO:teuthology.orchestra.run.smithi115.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-26T17:49:05.226 INFO:teuthology.orchestra.run.smithi115.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-26T17:49:05.243 INFO:teuthology.orchestra.run.smithi115.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.7 MB/s | 124 kB 00:00 2024-03-26T17:49:05.269 INFO:teuthology.orchestra.run.smithi115.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 4.4 MB/s | 187 kB 00:00 2024-03-26T17:49:05.461 INFO:teuthology.orchestra.run.smithi115.stdout:(88/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-03-26T17:49:05.528 INFO:teuthology.orchestra.run.smithi115.stdout:(89/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-26T17:49:05.590 INFO:teuthology.orchestra.run.smithi115.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-03-26T17:49:05.620 INFO:teuthology.orchestra.run.smithi115.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 476 kB/s | 187 kB 00:00 2024-03-26T17:49:05.644 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-03-26T17:49:05.820 INFO:teuthology.orchestra.run.smithi115.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 73 kB/s | 21 kB 00:00 2024-03-26T17:49:05.870 INFO:teuthology.orchestra.run.smithi115.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 91 kB/s | 25 kB 00:00 2024-03-26T17:49:06.596 INFO:teuthology.orchestra.run.smithi115.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 1.2 MB/s | 911 kB 00:00 2024-03-26T17:49:06.663 INFO:teuthology.orchestra.run.smithi115.stdout:(95/174): postgresql-13.14-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-03-26T17:49:06.704 INFO:teuthology.orchestra.run.smithi115.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-03-26T17:49:06.763 INFO:teuthology.orchestra.run.smithi115.stdout:(97/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 3.6 MB/s | 4.1 MB 00:01 2024-03-26T17:49:06.813 INFO:teuthology.orchestra.run.smithi115.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-03-26T17:49:06.872 INFO:teuthology.orchestra.run.smithi115.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-03-26T17:49:06.922 INFO:teuthology.orchestra.run.smithi115.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-03-26T17:49:06.981 INFO:teuthology.orchestra.run.smithi115.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 331 kB/s | 36 kB 00:00 2024-03-26T17:49:07.031 INFO:teuthology.orchestra.run.smithi115.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-03-26T17:49:07.089 INFO:teuthology.orchestra.run.smithi115.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 864 kB/s | 94 kB 00:00 2024-03-26T17:49:07.106 INFO:teuthology.orchestra.run.smithi115.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-03-26T17:49:07.257 INFO:teuthology.orchestra.run.smithi115.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-03-26T17:49:07.507 INFO:teuthology.orchestra.run.smithi115.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-03-26T17:49:07.574 INFO:teuthology.orchestra.run.smithi115.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 975 kB/s | 309 kB 00:00 2024-03-26T17:49:07.590 INFO:teuthology.orchestra.run.smithi115.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-03-26T17:49:07.666 INFO:teuthology.orchestra.run.smithi115.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-03-26T17:49:07.754 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:01 2024-03-26T17:49:07.766 INFO:teuthology.orchestra.run.smithi115.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-03-26T17:49:07.858 INFO:teuthology.orchestra.run.smithi115.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-03-26T17:49:07.875 INFO:teuthology.orchestra.run.smithi115.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-03-26T17:49:08.175 INFO:teuthology.orchestra.run.smithi115.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 477 kB/s | 151 kB 00:00 2024-03-26T17:49:08.301 INFO:teuthology.orchestra.run.smithi115.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.9 MB/s | 838 kB 00:00 2024-03-26T17:49:08.325 INFO:teuthology.orchestra.run.smithi115.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-03-26T17:49:08.384 INFO:teuthology.orchestra.run.smithi115.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 521 kB/s | 43 kB 00:00 2024-03-26T17:49:08.386 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.1 MB 00:00 2024-03-26T17:49:08.426 INFO:teuthology.orchestra.run.smithi115.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-03-26T17:49:08.468 INFO:teuthology.orchestra.run.smithi115.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-03-26T17:49:08.535 INFO:teuthology.orchestra.run.smithi115.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-03-26T17:49:08.627 INFO:teuthology.orchestra.run.smithi115.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-03-26T17:49:08.652 INFO:teuthology.orchestra.run.smithi115.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-03-26T17:49:08.727 INFO:teuthology.orchestra.run.smithi115.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-03-26T17:49:08.752 INFO:teuthology.orchestra.run.smithi115.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-03-26T17:49:08.862 INFO:teuthology.orchestra.run.smithi115.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-26T17:49:08.945 INFO:teuthology.orchestra.run.smithi115.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-03-26T17:49:09.054 INFO:teuthology.orchestra.run.smithi115.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.3 MB/s | 2.0 MB 00:00 2024-03-26T17:49:09.079 INFO:teuthology.orchestra.run.smithi115.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2024-03-26T17:49:09.154 INFO:teuthology.orchestra.run.smithi115.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-03-26T17:49:09.179 INFO:teuthology.orchestra.run.smithi115.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-03-26T17:49:09.280 INFO:teuthology.orchestra.run.smithi115.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-03-26T17:49:09.322 INFO:teuthology.orchestra.run.smithi115.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-03-26T17:49:09.347 INFO:teuthology.orchestra.run.smithi115.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 694 kB/s | 46 kB 00:00 2024-03-26T17:49:09.397 INFO:teuthology.orchestra.run.smithi115.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-03-26T17:49:09.447 INFO:teuthology.orchestra.run.smithi115.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-03-26T17:49:09.606 INFO:teuthology.orchestra.run.smithi115.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 104 kB/s | 16 kB 00:00 2024-03-26T17:49:09.690 INFO:teuthology.orchestra.run.smithi115.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 932 kB/s | 272 kB 00:00 2024-03-26T17:49:09.715 INFO:teuthology.orchestra.run.smithi115.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 829 kB/s | 90 kB 00:00 2024-03-26T17:49:09.790 INFO:teuthology.orchestra.run.smithi115.stdout:(138/174): postgresql-server-13.14-1.el9.x86_64 1.8 MB/s | 5.8 MB 00:03 2024-03-26T17:49:09.815 INFO:teuthology.orchestra.run.smithi115.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 225 kB/s | 28 kB 00:00 2024-03-26T17:49:09.841 INFO:teuthology.orchestra.run.smithi115.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-03-26T17:49:09.907 INFO:teuthology.orchestra.run.smithi115.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 646 kB/s | 59 kB 00:00 2024-03-26T17:49:09.933 INFO:teuthology.orchestra.run.smithi115.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-03-26T17:49:09.979 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-03-26T17:49:10.000 INFO:teuthology.orchestra.run.smithi115.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-03-26T17:49:10.042 INFO:teuthology.orchestra.run.smithi115.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-26T17:49:10.067 INFO:teuthology.orchestra.run.smithi115.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-26T17:49:10.092 INFO:teuthology.orchestra.run.smithi115.stdout:(146/174): python3-routes-2.5.1-5.el9.noarch.rp 625 kB/s | 188 kB 00:00 2024-03-26T17:49:10.142 INFO:teuthology.orchestra.run.smithi115.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-26T17:49:10.168 INFO:teuthology.orchestra.run.smithi115.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-03-26T17:49:10.243 INFO:teuthology.orchestra.run.smithi115.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-03-26T17:49:10.327 INFO:teuthology.orchestra.run.smithi115.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-03-26T17:49:10.444 INFO:teuthology.orchestra.run.smithi115.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.7 MB/s | 321 kB 00:00 2024-03-26T17:49:10.635 INFO:teuthology.orchestra.run.smithi115.stdout:(152/174): librados2-19.0.0-2311.g36051606.el9. 7.3 MB/s | 3.4 MB 00:00 2024-03-26T17:49:10.752 INFO:teuthology.orchestra.run.smithi115.stdout:(153/174): librbd1-19.0.0-2311.g36051606.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-03-26T17:49:10.869 INFO:teuthology.orchestra.run.smithi115.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 361 kB/s | 42 kB 00:00 2024-03-26T17:49:11.028 INFO:teuthology.orchestra.run.smithi115.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.5 MB/s | 248 kB 00:00 2024-03-26T17:49:11.086 INFO:teuthology.orchestra.run.smithi115.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-26T17:49:11.145 INFO:teuthology.orchestra.run.smithi115.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 627 kB/s | 36 kB 00:00 2024-03-26T17:49:11.203 INFO:teuthology.orchestra.run.smithi115.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-26T17:49:11.274 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-26T17:49:11.295 INFO:teuthology.orchestra.run.smithi115.stdout:(159/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.8 MB/s | 1.8 MB 00:00 2024-03-26T17:49:11.346 INFO:teuthology.orchestra.run.smithi115.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-03-26T17:49:11.388 INFO:teuthology.orchestra.run.smithi115.stdout:(161/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.4 MB/s | 2.3 MB 00:00 2024-03-26T17:49:11.413 INFO:teuthology.orchestra.run.smithi115.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 197 kB/s | 23 kB 00:00 2024-03-26T17:49:11.438 INFO:teuthology.orchestra.run.smithi115.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-03-26T17:49:11.505 INFO:teuthology.orchestra.run.smithi115.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 959 kB/s | 64 kB 00:00 2024-03-26T17:49:11.530 INFO:teuthology.orchestra.run.smithi115.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-26T17:49:11.555 INFO:teuthology.orchestra.run.smithi115.stdout:(166/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-03-26T17:49:11.580 INFO:teuthology.orchestra.run.smithi115.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 380 kB/s | 29 kB 00:00 2024-03-26T17:49:11.964 INFO:teuthology.orchestra.run.smithi115.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.3 MB/s | 2.2 MB 00:00 2024-03-26T17:49:12.031 INFO:teuthology.orchestra.run.smithi115.stdout:(169/174): openssl-3.0.7-27.el9.x86_64.rpm 2.4 MB/s | 1.2 MB 00:00 2024-03-26T17:49:12.090 INFO:teuthology.orchestra.run.smithi115.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 534 kB/s | 31 kB 00:00 2024-03-26T17:49:12.123 INFO:teuthology.orchestra.run.smithi115.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 2.9 MB/s | 465 kB 00:00 2024-03-26T17:49:12.207 INFO:teuthology.orchestra.run.smithi115.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 2.3 MB 00:00 2024-03-26T17:49:12.390 INFO:teuthology.orchestra.run.smithi115.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 278 kB/s | 74 kB 00:00 2024-03-26T17:49:12.482 INFO:teuthology.orchestra.run.smithi115.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 2.7 MB/s | 1.1 MB 00:00 2024-03-26T17:49:12.485 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:49:12.486 INFO:teuthology.orchestra.run.smithi115.stdout:Total 14 MB/s | 225 MB 00:15 2024-03-26T17:49:12.837 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-26T17:49:13.044 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-03-26T17:49:13.314 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-03-26T17:49:13.315 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-03-26T17:49:15.043 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-03-26T17:49:15.044 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-03-26T17:49:15.359 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:49:15.360 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:49:15.375 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T17:49:15.377 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T17:49:15.378 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T17:49:15.379 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T17:49:15.381 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T17:49:15.384 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T17:49:15.385 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T17:49:15.473 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-03-26T17:49:15.486 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-03-26T17:49:15.486 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-03-26T17:49:15.486 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-03-26T17:49:15.486 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-03-26T17:49:15.486 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T17:49:15.486 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:19.0.0-2311.g36051606.el9 ceph 6.6 k 2024-03-26T17:49:15.486 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.4 M 2024-03-26T17:49:15.486 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 890 k 2024-03-26T17:49:15.486 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2311.g36051606.el9 ceph 145 k 2024-03-26T17:49:15.486 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:19.0.0-2311.g36051606.el9 ceph 1.5 M 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 140 k 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 1.8 M 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 7.4 M 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 49 k 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 9.8 M 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:19.0.0-2311.g36051606.el9 ceph 48 M 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 264 k 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 750 k 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 221 k 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell x86_64 2:19.0.0-2311.g36051606.el9 ceph 27 k 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 18 k 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T17:49:15.487 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 32 k 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 746 k 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 127 k 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:19.0.0-2311.g36051606.el9 ceph 161 k 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:19.0.0-2311.g36051606.el9 ceph 321 k 2024-03-26T17:49:15.488 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 298 k 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 99 k 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 85 k 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 170 k 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T17:49:15.489 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T17:49:15.490 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T17:49:15.490 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T17:49:15.490 INFO:teuthology.orchestra.run.smithi049.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T17:49:15.490 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T17:49:15.490 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T17:49:15.490 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T17:49:15.490 INFO:teuthology.orchestra.run.smithi049.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T17:49:15.490 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.4 M 2024-03-26T17:49:15.490 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:49:15.490 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T17:49:15.491 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T17:49:15.492 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T17:49:15.492 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 21 M 2024-03-26T17:49:15.492 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 34 k 2024-03-26T17:49:15.492 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:19.0.0-2311.g36051606.el9 ceph 2.4 M 2024-03-26T17:49:15.492 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 248 k 2024-03-26T17:49:15.492 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:19.0.0-2311.g36051606.el9 ceph 4.7 M 2024-03-26T17:49:15.492 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:19.0.0-2311.g36051606.el9 ceph 17 M 2024-03-26T17:49:15.492 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 16 k 2024-03-26T17:49:15.492 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:19.0.0-2311.g36051606.el9 ceph 25 k 2024-03-26T17:49:15.492 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T17:49:15.493 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T17:49:15.493 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T17:49:15.493 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T17:49:15.493 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T17:49:15.493 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T17:49:15.493 INFO:teuthology.orchestra.run.smithi049.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T17:49:15.493 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T17:49:15.493 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T17:49:15.493 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:19.0.0-2311.g36051606.el9 ceph 163 k 2024-03-26T17:49:15.493 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T17:49:15.494 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T17:49:15.494 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T17:49:15.494 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T17:49:15.494 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 500 k 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.1 M 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T17:49:15.499 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:19.0.0-2311.g36051606.el9 ceph 45 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 131 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T17:49:15.500 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T17:49:15.501 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T17:49:15.502 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T17:49:15.503 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T17:49:15.504 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout:Installing weak dependencies: 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================ 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout:Install 150 Packages 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 24 Packages 2024-03-26T17:49:15.505 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:49:15.506 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 225 M 2024-03-26T17:49:15.506 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-03-26T17:49:16.257 INFO:teuthology.orchestra.run.smithi049.stdout:(1/174): ceph-19.0.0-2311.g36051606.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-03-26T17:49:16.305 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 8.2 kB/s | 1.7 kB 00:00 2024-03-26T17:49:16.466 INFO:teuthology.orchestra.run.smithi049.stdout:(2/174): ceph-fuse-19.0.0-2311.g36051606.el9.x8 4.2 MB/s | 890 kB 00:00 2024-03-26T17:49:16.516 INFO:teuthology.orchestra.run.smithi049.stdout:(3/174): ceph-immutable-object-cache-19.0.0-231 2.8 MB/s | 145 kB 00:00 2024-03-26T17:49:16.766 INFO:teuthology.orchestra.run.smithi049.stdout:(4/174): ceph-mds-19.0.0-2311.g36051606.el9.x86 9.4 MB/s | 2.4 MB 00:00 2024-03-26T17:49:16.950 INFO:teuthology.orchestra.run.smithi049.stdout:(5/174): ceph-mgr-19.0.0-2311.g36051606.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-03-26T17:49:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:(6/174): ceph-base-19.0.0-2311.g36051606.el9.x8 5.3 MB/s | 5.4 MB 00:01 2024-03-26T17:49:17.392 INFO:teuthology.orchestra.run.smithi049.stdout:(7/174): ceph-mon-19.0.0-2311.g36051606.el9.x86 11 MB/s | 4.7 MB 00:00 2024-03-26T17:49:18.534 INFO:teuthology.orchestra.run.smithi049.stdout:(8/174): ceph-radosgw-19.0.0-2311.g36051606.el9 8.6 MB/s | 9.8 MB 00:01 2024-03-26T17:49:18.601 INFO:teuthology.orchestra.run.smithi049.stdout:(9/174): ceph-selinux-19.0.0-2311.g36051606.el9 377 kB/s | 25 kB 00:00 2024-03-26T17:49:18.686 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:49:18.687 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:49:18.702 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T17:49:18.703 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T17:49:18.704 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T17:49:18.705 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T17:49:18.707 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T17:49:18.710 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T17:49:18.711 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T17:49:18.776 INFO:teuthology.orchestra.run.smithi049.stdout:(10/174): ceph-osd-19.0.0-2311.g36051606.el9.x8 10 MB/s | 17 MB 00:01 2024-03-26T17:49:18.791 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-03-26T17:49:18.812 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================================ 2024-03-26T17:49:18.812 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================================ 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.0.0-2311.g36051606.el9 ceph 6.6 k 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.4 M 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 890 k 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2311.g36051606.el9 ceph 145 k 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.0.0-2311.g36051606.el9 ceph 1.5 M 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 140 k 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 1.8 M 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 7.4 M 2024-03-26T17:49:18.813 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 49 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 9.8 M 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.0.0-2311.g36051606.el9 ceph 48 M 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 264 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 750 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 221 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell x86_64 2:19.0.0-2311.g36051606.el9 ceph 27 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 18 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 32 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 746 k 2024-03-26T17:49:18.814 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 127 k 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.0.0-2311.g36051606.el9 ceph 161 k 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.0.0-2311.g36051606.el9 ceph 321 k 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 298 k 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 99 k 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 85 k 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:49:18.815 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 170 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T17:49:18.816 INFO:teuthology.orchestra.run.smithi171.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.4 M 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-03-26T17:49:18.817 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 21 M 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 34 k 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.0.0-2311.g36051606.el9 ceph 2.4 M 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 248 k 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.0.0-2311.g36051606.el9 ceph 4.7 M 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.0.0-2311.g36051606.el9 ceph 17 M 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 16 k 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.0.0-2311.g36051606.el9 ceph 25 k 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T17:49:18.818 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.0.0-2311.g36051606.el9 ceph 163 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 500 k 2024-03-26T17:49:18.819 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.1 M 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T17:49:18.820 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.0.0-2311.g36051606.el9 ceph 45 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 131 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T17:49:18.821 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T17:49:18.822 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T17:49:18.823 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T17:49:18.824 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T17:49:18.825 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-03-26T17:49:18.826 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T17:49:18.826 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:49:18.826 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-03-26T17:49:18.826 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================================ 2024-03-26T17:49:18.826 INFO:teuthology.orchestra.run.smithi171.stdout:Install 150 Packages 2024-03-26T17:49:18.826 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 24 Packages 2024-03-26T17:49:18.826 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:49:18.826 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 225 M 2024-03-26T17:49:18.826 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-03-26T17:49:18.993 INFO:teuthology.orchestra.run.smithi049.stdout:(11/174): ceph-common-19.0.0-2311.g36051606.el9 7.1 MB/s | 21 MB 00:02 2024-03-26T17:49:19.018 INFO:teuthology.orchestra.run.smithi049.stdout:(12/174): cephfs-mirror-19.0.0-2311.g36051606.e 915 kB/s | 221 kB 00:00 2024-03-26T17:49:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:(13/174): cephfs-shell-19.0.0-2311.g36051606.el 360 kB/s | 27 kB 00:00 2024-03-26T17:49:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:(14/174): libcephfs-devel-19.0.0-2311.g36051606 420 kB/s | 32 kB 00:00 2024-03-26T17:49:19.177 INFO:teuthology.orchestra.run.smithi049.stdout:(15/174): libcephsqlite-19.0.0-2311.g36051606.e 1.9 MB/s | 163 kB 00:00 2024-03-26T17:49:19.211 INFO:teuthology.orchestra.run.smithi049.stdout:(16/174): libcephfs2-19.0.0-2311.g36051606.el9. 5.1 MB/s | 746 kB 00:00 2024-03-26T17:49:19.253 INFO:teuthology.orchestra.run.smithi049.stdout:(17/174): librados-devel-19.0.0-2311.g36051606. 1.6 MB/s | 127 kB 00:00 2024-03-26T17:49:19.320 INFO:teuthology.orchestra.run.smithi049.stdout:(18/174): libradosstriper1-19.0.0-2311.g3605160 4.5 MB/s | 500 kB 00:00 2024-03-26T17:49:19.420 INFO:teuthology.orchestra.run.smithi049.stdout:(19/174): python3-ceph-argparse-19.0.0-2311.g36 452 kB/s | 45 kB 00:00 2024-03-26T17:49:19.537 INFO:teuthology.orchestra.run.smithi049.stdout:(20/174): python3-ceph-common-19.0.0-2311.g3605 1.1 MB/s | 131 kB 00:00 2024-03-26T17:49:19.670 INFO:teuthology.orchestra.run.smithi049.stdout:(21/174): python3-cephfs-19.0.0-2311.g36051606. 1.2 MB/s | 161 kB 00:00 2024-03-26T17:49:19.738 INFO:teuthology.orchestra.run.smithi049.stdout:(22/174): librgw2-19.0.0-2311.g36051606.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-26T17:49:19.763 INFO:teuthology.orchestra.run.smithi049.stdout:(23/174): python3-rados-19.0.0-2311.g36051606.e 3.4 MB/s | 321 kB 00:00 2024-03-26T17:49:19.781 INFO:teuthology.orchestra.run.smithi171.stdout:(1/174): ceph-19.0.0-2311.g36051606.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-03-26T17:49:19.813 INFO:teuthology.orchestra.run.smithi049.stdout:(24/174): python3-rbd-19.0.0-2311.g36051606.el9 3.9 MB/s | 298 kB 00:00 2024-03-26T17:49:19.847 INFO:teuthology.orchestra.run.smithi049.stdout:(25/174): python3-rgw-19.0.0-2311.g36051606.el9 1.2 MB/s | 99 kB 00:00 2024-03-26T17:49:19.889 INFO:teuthology.orchestra.run.smithi049.stdout:(26/174): rbd-fuse-19.0.0-2311.g36051606.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-26T17:49:19.989 INFO:teuthology.orchestra.run.smithi049.stdout:(27/174): rbd-nbd-19.0.0-2311.g36051606.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-26T17:49:20.089 INFO:teuthology.orchestra.run.smithi049.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2311.g 340 kB/s | 34 kB 00:00 2024-03-26T17:49:20.198 INFO:teuthology.orchestra.run.smithi049.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2311.g3605160 1.3 MB/s | 140 kB 00:00 2024-03-26T17:49:20.257 INFO:teuthology.orchestra.run.smithi049.stdout:(30/174): rbd-mirror-19.0.0-2311.g36051606.el9. 7.5 MB/s | 3.1 MB 00:00 2024-03-26T17:49:20.515 INFO:teuthology.orchestra.run.smithi049.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2311.g36051 5.7 MB/s | 1.8 MB 00:00 2024-03-26T17:49:20.607 INFO:teuthology.orchestra.run.smithi049.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2311.g36 2.7 MB/s | 248 kB 00:00 2024-03-26T17:49:20.623 INFO:teuthology.orchestra.run.smithi171.stdout:(2/174): ceph-fuse-19.0.0-2311.g36051606.el9.x8 1.0 MB/s | 890 kB 00:00 2024-03-26T17:49:20.682 INFO:teuthology.orchestra.run.smithi049.stdout:(33/174): ceph-mgr-rook-19.0.0-2311.g36051606.e 658 kB/s | 49 kB 00:00 2024-03-26T17:49:20.766 INFO:teuthology.orchestra.run.smithi049.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2311.g3 198 kB/s | 16 kB 00:00 2024-03-26T17:49:20.773 INFO:teuthology.orchestra.run.smithi171.stdout:(3/174): ceph-immutable-object-cache-19.0.0-231 968 kB/s | 145 kB 00:00 2024-03-26T17:49:20.850 INFO:teuthology.orchestra.run.smithi049.stdout:(35/174): ceph-volume-19.0.0-2311.g36051606.el9 3.1 MB/s | 264 kB 00:00 2024-03-26T17:49:21.033 INFO:teuthology.orchestra.run.smithi049.stdout:(36/174): cephadm-19.0.0-2311.g36051606.el9.noa 4.0 MB/s | 750 kB 00:00 2024-03-26T17:49:21.125 INFO:teuthology.orchestra.run.smithi049.stdout:(37/174): cephfs-top-19.0.0-2311.g36051606.el9. 200 kB/s | 18 kB 00:00 2024-03-26T17:49:21.275 INFO:teuthology.orchestra.run.smithi049.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 662 kB 00:00 2024-03-26T17:49:21.334 INFO:teuthology.orchestra.run.smithi049.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-03-26T17:49:21.426 INFO:teuthology.orchestra.run.smithi049.stdout:(40/174): ceph-mgr-diskprediction-local-19.0.0- 6.3 MB/s | 7.4 MB 00:01 2024-03-26T17:49:21.451 INFO:teuthology.orchestra.run.smithi049.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 906 kB/s | 106 kB 00:00 2024-03-26T17:49:21.476 INFO:teuthology.orchestra.run.smithi049.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-26T17:49:21.502 INFO:teuthology.orchestra.run.smithi049.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2024-03-26T17:49:21.569 INFO:teuthology.orchestra.run.smithi049.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-03-26T17:49:21.594 INFO:teuthology.orchestra.run.smithi049.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 192 kB/s | 17 kB 00:00 2024-03-26T17:49:21.619 INFO:teuthology.orchestra.run.smithi049.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2024-03-26T17:49:21.644 INFO:teuthology.orchestra.run.smithi049.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-26T17:49:21.670 INFO:teuthology.orchestra.run.smithi049.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 884 kB/s | 44 kB 00:00 2024-03-26T17:49:21.703 INFO:teuthology.orchestra.run.smithi049.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-26T17:49:21.737 INFO:teuthology.orchestra.run.smithi049.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 578 kB 00:00 2024-03-26T17:49:21.740 INFO:teuthology.orchestra.run.smithi171.stdout:(4/174): ceph-mds-19.0.0-2311.g36051606.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-03-26T17:49:21.762 INFO:teuthology.orchestra.run.smithi049.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 808 kB/s | 46 kB 00:00 2024-03-26T17:49:21.788 INFO:teuthology.orchestra.run.smithi049.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-26T17:49:21.813 INFO:teuthology.orchestra.run.smithi049.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T17:49:21.838 INFO:teuthology.orchestra.run.smithi049.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-26T17:49:21.863 INFO:teuthology.orchestra.run.smithi049.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-03-26T17:49:21.873 INFO:teuthology.orchestra.run.smithi171.stdout:(5/174): ceph-base-19.0.0-2311.g36051606.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-03-26T17:49:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-03-26T17:49:21.964 INFO:teuthology.orchestra.run.smithi049.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-26T17:49:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T17:49:22.005 INFO:teuthology.orchestra.run.smithi049.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.3 MB/s | 1.0 MB 00:00 2024-03-26T17:49:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T17:49:22.031 INFO:teuthology.orchestra.run.smithi049.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-26T17:49:22.058 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:49:22.067 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:49:22.074 INFO:teuthology.orchestra.run.smithi171.stdout:(6/174): ceph-common-19.0.0-2311.g36051606.el9. 8.0 MB/s | 21 MB 00:02 2024-03-26T17:49:22.107 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T17:49:22.161 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T17:49:22.195 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T17:49:22.212 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T17:49:22.223 INFO:teuthology.orchestra.run.smithi049.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-26T17:49:22.248 INFO:teuthology.orchestra.run.smithi049.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 113 kB/s | 24 kB 00:00 2024-03-26T17:49:22.254 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T17:49:22.266 INFO:teuthology.orchestra.run.smithi171.stdout:(7/174): ceph-mgr-19.0.0-2311.g36051606.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-03-26T17:49:22.307 INFO:teuthology.orchestra.run.smithi049.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 311 kB/s | 18 kB 00:00 2024-03-26T17:49:22.324 INFO:teuthology.orchestra.run.smithi049.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-26T17:49:22.374 INFO:teuthology.orchestra.run.smithi049.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 474 kB/s | 32 kB 00:00 2024-03-26T17:49:22.566 INFO:teuthology.orchestra.run.smithi049.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-26T17:49:22.591 INFO:teuthology.orchestra.run.smithi049.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2024-03-26T17:49:22.633 INFO:teuthology.orchestra.run.smithi049.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-03-26T17:49:22.658 INFO:teuthology.orchestra.run.smithi049.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-26T17:49:22.692 INFO:teuthology.orchestra.run.smithi049.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 708 kB/s | 41 kB 00:00 2024-03-26T17:49:22.717 INFO:teuthology.orchestra.run.smithi049.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-26T17:49:22.759 INFO:teuthology.orchestra.run.smithi049.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-26T17:49:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T17:49:22.801 INFO:teuthology.orchestra.run.smithi049.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-03-26T17:49:22.821 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:49:22.827 INFO:teuthology.orchestra.run.smithi049.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 706 kB/s | 47 kB 00:00 2024-03-26T17:49:22.852 INFO:teuthology.orchestra.run.smithi049.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 851 kB/s | 43 kB 00:00 2024-03-26T17:49:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:49:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:49:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:49:23.044 INFO:teuthology.orchestra.run.smithi049.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 201 kB/s | 39 kB 00:00 2024-03-26T17:49:23.060 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:49:23.099 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T17:49:23.266 INFO:teuthology.orchestra.run.smithi171.stdout:(8/174): ceph-mon-19.0.0-2311.g36051606.el9.x86 3.4 MB/s | 4.7 MB 00:01 2024-03-26T17:49:23.267 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T17:49:23.286 INFO:teuthology.orchestra.run.smithi049.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-03-26T17:49:23.311 INFO:teuthology.orchestra.run.smithi049.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 155 kB/s | 41 kB 00:00 2024-03-26T17:49:23.316 INFO:teuthology.orchestra.run.smithi171.stdout:(9/174): ceph-selinux-19.0.0-2311.g36051606.el9 504 kB/s | 25 kB 00:00 2024-03-26T17:49:23.324 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T17:49:23.352 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:49:23.456 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:49:23.478 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:49:23.500 INFO:teuthology.orchestra.run.smithi171.stdout:(10/174): ceph-osd-19.0.0-2311.g36051606.el9.x8 12 MB/s | 17 MB 00:01 2024-03-26T17:49:23.542 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:49:23.558 INFO:teuthology.orchestra.run.smithi171.stdout:(11/174): cephfs-mirror-19.0.0-2311.g36051606.e 3.7 MB/s | 221 kB 00:00 2024-03-26T17:49:23.582 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T17:49:23.592 INFO:teuthology.orchestra.run.smithi171.stdout:(12/174): cephfs-shell-19.0.0-2311.g36051606.el 812 kB/s | 27 kB 00:00 2024-03-26T17:49:23.605 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:49:23.619 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:49:23.625 INFO:teuthology.orchestra.run.smithi171.stdout:(13/174): libcephfs-devel-19.0.0-2311.g36051606 954 kB/s | 32 kB 00:00 2024-03-26T17:49:23.709 INFO:teuthology.orchestra.run.smithi171.stdout:(14/174): libcephfs2-19.0.0-2311.g36051606.el9. 8.8 MB/s | 746 kB 00:00 2024-03-26T17:49:23.745 INFO:teuthology.orchestra.run.smithi049.stdout:(77/174): ceph-test-19.0.0-2311.g36051606.el9.x 9.3 MB/s | 48 MB 00:05 2024-03-26T17:49:23.759 INFO:teuthology.orchestra.run.smithi171.stdout:(15/174): libcephsqlite-19.0.0-2311.g36051606.e 3.2 MB/s | 163 kB 00:00 2024-03-26T17:49:23.770 INFO:teuthology.orchestra.run.smithi049.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 620 kB/s | 300 kB 00:00 2024-03-26T17:49:23.795 INFO:teuthology.orchestra.run.smithi049.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 116 kB/s | 56 kB 00:00 2024-03-26T17:49:23.809 INFO:teuthology.orchestra.run.smithi171.stdout:(16/174): librados-devel-19.0.0-2311.g36051606. 2.5 MB/s | 127 kB 00:00 2024-03-26T17:49:23.813 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:49:23.820 INFO:teuthology.orchestra.run.smithi049.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 842 kB/s | 63 kB 00:00 2024-03-26T17:49:23.844 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:49:23.845 INFO:teuthology.orchestra.run.smithi049.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-26T17:49:23.870 INFO:teuthology.orchestra.run.smithi049.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-26T17:49:23.875 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T17:49:23.876 INFO:teuthology.orchestra.run.smithi171.stdout:(17/174): libradosstriper1-19.0.0-2311.g3605160 7.3 MB/s | 500 kB 00:00 2024-03-26T17:49:23.896 INFO:teuthology.orchestra.run.smithi049.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-26T17:49:23.921 INFO:teuthology.orchestra.run.smithi049.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-26T17:49:23.955 INFO:teuthology.orchestra.run.smithi049.stdout:(85/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-26T17:49:23.980 INFO:teuthology.orchestra.run.smithi049.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-26T17:49:23.981 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T17:49:24.005 INFO:teuthology.orchestra.run.smithi049.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 187 kB 00:00 2024-03-26T17:49:24.055 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:49:24.105 INFO:teuthology.orchestra.run.smithi049.stdout:(88/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-03-26T17:49:24.115 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:49:24.164 INFO:teuthology.orchestra.run.smithi049.stdout:(89/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-03-26T17:49:24.184 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi049.stdout:(90/174): jq-1.6-16.el9.x86_64.rpm 897 kB/s | 187 kB 00:00 2024-03-26T17:49:24.227 INFO:teuthology.orchestra.run.smithi171.stdout:(18/174): librgw2-19.0.0-2311.g36051606.el9.x86 15 MB/s | 5.1 MB 00:00 2024-03-26T17:49:24.231 INFO:teuthology.orchestra.run.smithi049.stdout:(91/174): python3-requests-2.25.1-8.el9.noarch. 993 kB/s | 125 kB 00:00 2024-03-26T17:49:24.241 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:49:24.269 INFO:teuthology.orchestra.run.smithi171.stdout:(19/174): python3-ceph-argparse-19.0.0-2311.g36 1.1 MB/s | 45 kB 00:00 2024-03-26T17:49:24.311 INFO:teuthology.orchestra.run.smithi171.stdout:(20/174): python3-ceph-common-19.0.0-2311.g3605 3.1 MB/s | 131 kB 00:00 2024-03-26T17:49:24.315 INFO:teuthology.orchestra.run.smithi049.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 143 kB/s | 21 kB 00:00 2024-03-26T17:49:24.332 INFO:teuthology.orchestra.run.smithi049.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 179 kB/s | 25 kB 00:00 2024-03-26T17:49:24.384 INFO:teuthology.orchestra.run.smithi171.stdout:(21/174): python3-cephfs-19.0.0-2311.g36051606. 2.2 MB/s | 161 kB 00:00 2024-03-26T17:49:24.444 INFO:teuthology.orchestra.run.smithi171.stdout:(22/174): python3-rados-19.0.0-2311.g36051606.e 5.3 MB/s | 321 kB 00:00 2024-03-26T17:49:24.462 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:49:24.503 INFO:teuthology.orchestra.run.smithi171.stdout:(23/174): python3-rbd-19.0.0-2311.g36051606.el9 5.0 MB/s | 298 kB 00:00 2024-03-26T17:49:24.532 INFO:teuthology.orchestra.run.smithi049.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 4.5 MB/s | 911 kB 00:00 2024-03-26T17:49:24.553 INFO:teuthology.orchestra.run.smithi171.stdout:(24/174): python3-rgw-19.0.0-2311.g36051606.el9 1.9 MB/s | 99 kB 00:00 2024-03-26T17:49:24.570 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T17:49:24.591 INFO:teuthology.orchestra.run.smithi049.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-03-26T17:49:24.595 INFO:teuthology.orchestra.run.smithi171.stdout:(25/174): rbd-fuse-19.0.0-2311.g36051606.el9.x8 2.0 MB/s | 85 kB 00:00 2024-03-26T17:49:24.624 INFO:teuthology.orchestra.run.smithi049.stdout:(96/174): postgresql-13.14-1.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-03-26T17:49:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T17:49:24.691 INFO:teuthology.orchestra.run.smithi049.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-03-26T17:49:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T17:49:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T17:49:24.758 INFO:teuthology.orchestra.run.smithi049.stdout:(98/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 7.9 MB/s | 4.1 MB 00:00 2024-03-26T17:49:24.759 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T17:49:24.784 INFO:teuthology.orchestra.run.smithi049.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-03-26T17:49:24.817 INFO:teuthology.orchestra.run.smithi049.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-03-26T17:49:24.833 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T17:49:24.837 INFO:teuthology.orchestra.run.smithi171.stdout:(26/174): rbd-mirror-19.0.0-2311.g36051606.el9. 13 MB/s | 3.1 MB 00:00 2024-03-26T17:49:24.842 INFO:teuthology.orchestra.run.smithi049.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 613 kB/s | 36 kB 00:00 2024-03-26T17:49:24.876 INFO:teuthology.orchestra.run.smithi049.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-03-26T17:49:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T17:49:24.887 INFO:teuthology.orchestra.run.smithi171.stdout:(27/174): rbd-nbd-19.0.0-2311.g36051606.el9.x86 3.4 MB/s | 170 kB 00:00 2024-03-26T17:49:24.901 INFO:teuthology.orchestra.run.smithi049.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T17:49:24.946 INFO:teuthology.orchestra.run.smithi171.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2311.g 581 kB/s | 34 kB 00:00 2024-03-26T17:49:24.989 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T17:49:24.997 INFO:teuthology.orchestra.run.smithi171.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2311.g3605160 2.7 MB/s | 140 kB 00:00 2024-03-26T17:49:25.010 INFO:teuthology.orchestra.run.smithi049.stdout:(104/174): postgresql-server-13.14-1.el9.x86_64 14 MB/s | 5.8 MB 00:00 2024-03-26T17:49:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T17:49:25.035 INFO:teuthology.orchestra.run.smithi049.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 140 kB/s | 22 kB 00:00 2024-03-26T17:49:25.068 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T17:49:25.099 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T17:49:25.127 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T17:49:25.147 INFO:teuthology.orchestra.run.smithi171.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2311.g36051 12 MB/s | 1.8 MB 00:00 2024-03-26T17:49:25.194 INFO:teuthology.orchestra.run.smithi049.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-03-26T17:49:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T17:49:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 39/198 2024-03-26T17:49:25.267 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 40/198 2024-03-26T17:49:25.296 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 41/198 2024-03-26T17:49:25.364 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T17:49:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T17:49:25.514 INFO:teuthology.orchestra.run.smithi171.stdout:(31/174): ceph-radosgw-19.0.0-2311.g36051606.el 3.0 MB/s | 9.8 MB 00:03 2024-03-26T17:49:25.617 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T17:49:25.631 INFO:teuthology.orchestra.run.smithi171.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2311.g36 2.1 MB/s | 248 kB 00:00 2024-03-26T17:49:25.635 INFO:teuthology.orchestra.run.smithi049.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 44 kB/s | 19 kB 00:00 2024-03-26T17:49:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T17:49:25.731 INFO:teuthology.orchestra.run.smithi171.stdout:(33/174): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-03-26T17:49:25.764 INFO:teuthology.orchestra.run.smithi171.stdout:(34/174): ceph-mgr-rook-19.0.0-2311.g36051606.e 369 kB/s | 49 kB 00:00 2024-03-26T17:49:25.789 INFO:teuthology.orchestra.run.smithi171.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2311.g3 282 kB/s | 16 kB 00:00 2024-03-26T17:49:25.873 INFO:teuthology.orchestra.run.smithi171.stdout:(36/174): cephadm-19.0.0-2311.g36051606.el9.noa 8.8 MB/s | 750 kB 00:00 2024-03-26T17:49:25.899 INFO:teuthology.orchestra.run.smithi171.stdout:(37/174): ceph-volume-19.0.0-2311.g36051606.el9 1.9 MB/s | 264 kB 00:00 2024-03-26T17:49:25.924 INFO:teuthology.orchestra.run.smithi171.stdout:(38/174): cephfs-top-19.0.0-2311.g36051606.el9. 365 kB/s | 18 kB 00:00 2024-03-26T17:49:25.927 INFO:teuthology.orchestra.run.smithi049.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 346 kB/s | 309 kB 00:00 2024-03-26T17:49:26.053 INFO:teuthology.orchestra.run.smithi049.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 526 kB/s | 548 kB 00:01 2024-03-26T17:49:26.074 INFO:teuthology.orchestra.run.smithi171.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 662 kB 00:00 2024-03-26T17:49:26.116 INFO:teuthology.orchestra.run.smithi171.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2024-03-26T17:49:26.128 INFO:teuthology.orchestra.run.smithi049.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-03-26T17:49:26.141 INFO:teuthology.orchestra.run.smithi171.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 920 kB 00:00 2024-03-26T17:49:26.167 INFO:teuthology.orchestra.run.smithi171.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-26T17:49:26.192 INFO:teuthology.orchestra.run.smithi171.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 635 kB/s | 32 kB 00:00 2024-03-26T17:49:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T17:49:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T17:49:26.300 INFO:teuthology.orchestra.run.smithi171.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-03-26T17:49:26.326 INFO:teuthology.orchestra.run.smithi171.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 129 kB/s | 17 kB 00:00 2024-03-26T17:49:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T17:49:26.359 INFO:teuthology.orchestra.run.smithi171.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 305 kB/s | 10 kB 00:00 2024-03-26T17:49:26.384 INFO:teuthology.orchestra.run.smithi171.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-26T17:49:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:49:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:49:26.403 INFO:teuthology.orchestra.run.smithi049.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-03-26T17:49:26.410 INFO:teuthology.orchestra.run.smithi171.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-26T17:49:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T17:49:26.435 INFO:teuthology.orchestra.run.smithi171.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-26T17:49:26.460 INFO:teuthology.orchestra.run.smithi171.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-03-26T17:49:26.467 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T17:49:26.477 INFO:teuthology.orchestra.run.smithi171.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-26T17:49:26.502 INFO:teuthology.orchestra.run.smithi171.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-26T17:49:26.505 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 53/198 2024-03-26T17:49:26.528 INFO:teuthology.orchestra.run.smithi171.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T17:49:26.534 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T17:49:26.545 INFO:teuthology.orchestra.run.smithi049.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 161 kB/s | 67 kB 00:00 2024-03-26T17:49:26.553 INFO:teuthology.orchestra.run.smithi171.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-26T17:49:26.570 INFO:teuthology.orchestra.run.smithi049.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 905 kB/s | 151 kB 00:00 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi171.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-03-26T17:49:26.755 INFO:teuthology.orchestra.run.smithi171.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-03-26T17:49:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T17:49:26.789 INFO:teuthology.orchestra.run.smithi171.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-03-26T17:49:26.803 INFO:teuthology.orchestra.run.smithi049.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-03-26T17:49:26.805 INFO:teuthology.orchestra.run.smithi171.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.6 MB/s | 78 kB 00:00 2024-03-26T17:49:26.896 INFO:teuthology.orchestra.run.smithi049.stdout:(115/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-03-26T17:49:26.930 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T17:49:26.988 INFO:teuthology.orchestra.run.smithi049.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 236 kB/s | 43 kB 00:00 2024-03-26T17:49:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T17:49:27.031 INFO:teuthology.orchestra.run.smithi171.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 5.8 MB 00:00 2024-03-26T17:49:27.038 INFO:teuthology.orchestra.run.smithi049.stdout:(117/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 838 kB 00:00 2024-03-26T17:49:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T17:49:27.056 INFO:teuthology.orchestra.run.smithi171.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 97 kB/s | 24 kB 00:00 2024-03-26T17:49:27.063 INFO:teuthology.orchestra.run.smithi049.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 193 kB/s | 32 kB 00:00 2024-03-26T17:49:27.089 INFO:teuthology.orchestra.run.smithi171.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 544 kB/s | 18 kB 00:00 2024-03-26T17:49:27.115 INFO:teuthology.orchestra.run.smithi171.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-26T17:49:27.140 INFO:teuthology.orchestra.run.smithi171.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-26T17:49:27.146 INFO:teuthology.orchestra.run.smithi049.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 94 kB/s | 15 kB 00:00 2024-03-26T17:49:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T17:49:27.205 INFO:teuthology.orchestra.run.smithi049.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-03-26T17:49:27.234 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T17:49:27.239 INFO:teuthology.orchestra.run.smithi049.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.0 MB/s | 358 kB 00:00 2024-03-26T17:49:27.282 INFO:teuthology.orchestra.run.smithi171.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-03-26T17:49:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T17:49:27.307 INFO:teuthology.orchestra.run.smithi171.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-26T17:49:27.339 INFO:teuthology.orchestra.run.smithi049.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-03-26T17:49:27.341 INFO:teuthology.orchestra.run.smithi171.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 174 kB 00:00 2024-03-26T17:49:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T17:49:27.366 INFO:teuthology.orchestra.run.smithi171.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 69 kB 00:00 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi049.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 201 kB/s | 35 kB 00:00 2024-03-26T17:49:27.385 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi171.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 822 kB/s | 41 kB 00:00 2024-03-26T17:49:27.406 INFO:teuthology.orchestra.run.smithi049.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.2 MB/s | 203 kB 00:00 2024-03-26T17:49:27.417 INFO:teuthology.orchestra.run.smithi171.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-26T17:49:27.421 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T17:49:27.442 INFO:teuthology.orchestra.run.smithi171.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-26T17:49:27.468 INFO:teuthology.orchestra.run.smithi171.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-26T17:49:27.493 INFO:teuthology.orchestra.run.smithi171.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2024-03-26T17:49:27.518 INFO:teuthology.orchestra.run.smithi171.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 856 kB/s | 43 kB 00:00 2024-03-26T17:49:27.548 INFO:teuthology.orchestra.run.smithi049.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 866 kB/s | 144 kB 00:00 2024-03-26T17:49:27.552 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T17:49:27.565 INFO:teuthology.orchestra.run.smithi049.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 67 kB/s | 11 kB 00:00 2024-03-26T17:49:27.587 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T17:49:27.630 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T17:49:27.635 INFO:teuthology.orchestra.run.smithi171.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 334 kB/s | 39 kB 00:00 2024-03-26T17:49:27.673 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T17:49:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.5 MB/s | 2.0 MB 00:00 2024-03-26T17:49:27.757 INFO:teuthology.orchestra.run.smithi049.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 121 kB/s | 23 kB 00:00 2024-03-26T17:49:27.807 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T17:49:27.853 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T17:49:27.858 INFO:teuthology.orchestra.run.smithi049.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 129 kB/s | 19 kB 00:00 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi049.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 131 kB/s | 20 kB 00:00 2024-03-26T17:49:27.926 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T17:49:27.944 INFO:teuthology.orchestra.run.smithi171.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-26T17:49:27.969 INFO:teuthology.orchestra.run.smithi171.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 124 kB/s | 41 kB 00:00 2024-03-26T17:49:27.994 INFO:teuthology.orchestra.run.smithi171.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-26T17:49:28.019 INFO:teuthology.orchestra.run.smithi171.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-26T17:49:28.021 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi171.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T17:49:28.070 INFO:teuthology.orchestra.run.smithi171.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-03-26T17:49:28.095 INFO:teuthology.orchestra.run.smithi171.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-26T17:49:28.100 INFO:teuthology.orchestra.run.smithi049.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-03-26T17:49:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T17:49:28.117 INFO:teuthology.orchestra.run.smithi049.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 222 kB/s | 46 kB 00:00 2024-03-26T17:49:28.120 INFO:teuthology.orchestra.run.smithi171.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-26T17:49:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T17:49:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T17:49:28.162 INFO:teuthology.orchestra.run.smithi171.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-26T17:49:28.188 INFO:teuthology.orchestra.run.smithi171.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-26T17:49:28.213 INFO:teuthology.orchestra.run.smithi171.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-26T17:49:28.223 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T17:49:28.238 INFO:teuthology.orchestra.run.smithi171.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-26T17:49:28.259 INFO:teuthology.orchestra.run.smithi049.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 498 kB/s | 79 kB 00:00 2024-03-26T17:49:28.276 INFO:teuthology.orchestra.run.smithi049.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 364 kB/s | 58 kB 00:00 2024-03-26T17:49:28.301 INFO:teuthology.orchestra.run.smithi049.stdout:(135/174): python3-jaraco-classes-3.2.1-5.el9.n 23 kB/s | 18 kB 00:00 2024-03-26T17:49:28.351 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T17:49:28.418 INFO:teuthology.orchestra.run.smithi049.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-03-26T17:49:28.443 INFO:teuthology.orchestra.run.smithi049.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 98 kB/s | 16 kB 00:00 2024-03-26T17:49:28.460 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T17:49:28.468 INFO:teuthology.orchestra.run.smithi049.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 538 kB/s | 90 kB 00:00 2024-03-26T17:49:28.501 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T17:49:28.550 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T17:49:28.602 INFO:teuthology.orchestra.run.smithi049.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 153 kB/s | 28 kB 00:00 2024-03-26T17:49:28.605 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T17:49:28.627 INFO:teuthology.orchestra.run.smithi049.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 167 kB/s | 31 kB 00:00 2024-03-26T17:49:28.652 INFO:teuthology.orchestra.run.smithi049.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-03-26T17:49:28.664 INFO:teuthology.orchestra.run.smithi171.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 416 kB/s | 187 kB 00:00 2024-03-26T17:49:28.761 INFO:teuthology.orchestra.run.smithi049.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 372 kB/s | 59 kB 00:00 2024-03-26T17:49:28.786 INFO:teuthology.orchestra.run.smithi049.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 225 kB/s | 36 kB 00:00 2024-03-26T17:49:28.787 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T17:49:28.811 INFO:teuthology.orchestra.run.smithi049.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 321 kB/s | 51 kB 00:00 2024-03-26T17:49:28.838 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T17:49:28.847 INFO:teuthology.orchestra.run.smithi171.stdout:(88/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-03-26T17:49:28.906 INFO:teuthology.orchestra.run.smithi171.stdout:(89/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-03-26T17:49:28.945 INFO:teuthology.orchestra.run.smithi049.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-03-26T17:49:28.956 INFO:teuthology.orchestra.run.smithi171.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-26T17:49:28.968 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T17:49:28.970 INFO:teuthology.orchestra.run.smithi049.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 488 kB/s | 90 kB 00:00 2024-03-26T17:49:29.004 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T17:49:29.023 INFO:teuthology.orchestra.run.smithi171.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 184 kB/s | 21 kB 00:00 2024-03-26T17:49:29.035 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T17:49:29.065 INFO:teuthology.orchestra.run.smithi171.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-03-26T17:49:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T17:49:29.094 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T17:49:29.104 INFO:teuthology.orchestra.run.smithi049.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 126 kB/s | 20 kB 00:00 2024-03-26T17:49:29.129 INFO:teuthology.orchestra.run.smithi049.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.3 MB/s | 427 kB 00:00 2024-03-26T17:49:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T17:49:29.163 INFO:teuthology.orchestra.run.smithi049.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 997 kB/s | 192 kB 00:00 2024-03-26T17:49:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T17:49:29.208 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T17:49:29.238 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T17:49:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T17:49:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T17:49:29.421 INFO:teuthology.orchestra.run.smithi049.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T17:49:29.457 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T17:49:29.480 INFO:teuthology.orchestra.run.smithi049.stdout:(151/174): librados2-19.0.0-2311.g36051606.el9. 9.7 MB/s | 3.4 MB 00:00 2024-03-26T17:49:29.505 INFO:teuthology.orchestra.run.smithi049.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-03-26T17:49:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T17:49:29.639 INFO:teuthology.orchestra.run.smithi049.stdout:(153/174): librbd1-19.0.0-2311.g36051606.el9.x8 6.6 MB/s | 3.1 MB 00:00 2024-03-26T17:49:29.707 INFO:teuthology.orchestra.run.smithi171.stdout:(93/174): ceph-test-19.0.0-2311.g36051606.el9.x 7.5 MB/s | 48 MB 00:06 2024-03-26T17:49:29.749 INFO:teuthology.orchestra.run.smithi171.stdout:(94/174): postgresql-13.14-1.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-03-26T17:49:29.772 INFO:teuthology.orchestra.run.smithi049.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 316 kB/s | 42 kB 00:00 2024-03-26T17:49:29.807 INFO:teuthology.orchestra.run.smithi171.stdout:(95/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.3 MB/s | 4.1 MB 00:00 2024-03-26T17:49:29.890 INFO:teuthology.orchestra.run.smithi049.stdout:(155/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.8 MB/s | 1.8 MB 00:00 2024-03-26T17:49:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T17:49:29.916 INFO:teuthology.orchestra.run.smithi171.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 798 kB/s | 133 kB 00:00 2024-03-26T17:49:29.940 INFO:teuthology.orchestra.run.smithi049.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.5 MB/s | 248 kB 00:00 2024-03-26T17:49:29.964 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T17:49:29.966 INFO:teuthology.orchestra.run.smithi171.stdout:(97/174): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-03-26T17:49:29.990 INFO:teuthology.orchestra.run.smithi049.stdout:(157/174): NetworkManager-1.47.2-1.el9.x86_64.r 4.5 MB/s | 2.3 MB 00:00 2024-03-26T17:49:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T17:49:30.015 INFO:teuthology.orchestra.run.smithi049.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 568 kB/s | 71 kB 00:00 2024-03-26T17:49:30.041 INFO:teuthology.orchestra.run.smithi049.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 362 kB/s | 36 kB 00:00 2024-03-26T17:49:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T17:49:30.092 INFO:teuthology.orchestra.run.smithi171.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-03-26T17:49:30.117 INFO:teuthology.orchestra.run.smithi171.stdout:(99/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-03-26T17:49:30.140 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T17:49:30.149 INFO:teuthology.orchestra.run.smithi049.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 213 kB/s | 23 kB 00:00 2024-03-26T17:49:30.174 INFO:teuthology.orchestra.run.smithi049.stdout:(161/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.0 MB/s | 192 kB 00:00 2024-03-26T17:49:30.190 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T17:49:30.200 INFO:teuthology.orchestra.run.smithi049.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.4 MB/s | 256 kB 00:00 2024-03-26T17:49:30.200 INFO:teuthology.orchestra.run.smithi171.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 432 kB/s | 36 kB 00:00 2024-03-26T17:49:30.267 INFO:teuthology.orchestra.run.smithi171.stdout:(101/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-03-26T17:49:30.300 INFO:teuthology.orchestra.run.smithi049.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 722 kB/s | 108 kB 00:00 2024-03-26T17:49:30.325 INFO:teuthology.orchestra.run.smithi049.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 155 kB 00:00 2024-03-26T17:49:30.351 INFO:teuthology.orchestra.run.smithi171.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 315 kB/s | 47 kB 00:00 2024-03-26T17:49:30.368 INFO:teuthology.orchestra.run.smithi049.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 809 kB/s | 136 kB 00:00 2024-03-26T17:49:30.417 INFO:teuthology.orchestra.run.smithi049.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 545 kB/s | 64 kB 00:00 2024-03-26T17:49:30.418 INFO:teuthology.orchestra.run.smithi171.stdout:(103/174): postgresql-server-13.14-1.el9.x86_64 9.5 MB/s | 5.8 MB 00:00 2024-03-26T17:49:30.442 INFO:teuthology.orchestra.run.smithi049.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 245 kB/s | 29 kB 00:00 2024-03-26T17:49:30.443 INFO:teuthology.orchestra.run.smithi171.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 534 kB/s | 94 kB 00:00 2024-03-26T17:49:30.510 INFO:teuthology.orchestra.run.smithi049.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 8.6 MB/s | 1.2 MB 00:00 2024-03-26T17:49:30.535 INFO:teuthology.orchestra.run.smithi171.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 121 kB/s | 22 kB 00:00 2024-03-26T17:49:30.618 INFO:teuthology.orchestra.run.smithi049.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 13 MB/s | 2.3 MB 00:00 2024-03-26T17:49:30.643 INFO:teuthology.orchestra.run.smithi049.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.4 MB/s | 465 kB 00:00 2024-03-26T17:49:30.718 INFO:teuthology.orchestra.run.smithi049.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 311 kB/s | 31 kB 00:00 2024-03-26T17:49:30.752 INFO:teuthology.orchestra.run.smithi171.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-03-26T17:49:30.777 INFO:teuthology.orchestra.run.smithi171.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-03-26T17:49:30.802 INFO:teuthology.orchestra.run.smithi049.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.6 MB/s | 2.2 MB 00:00 2024-03-26T17:49:30.803 INFO:teuthology.orchestra.run.smithi171.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-03-26T17:49:30.844 INFO:teuthology.orchestra.run.smithi049.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.4 MB/s | 1.1 MB 00:00 2024-03-26T17:49:30.845 INFO:teuthology.orchestra.run.smithi171.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 625 kB/s | 26 kB 00:00 2024-03-26T17:49:30.861 INFO:teuthology.orchestra.run.smithi049.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 523 kB/s | 74 kB 00:00 2024-03-26T17:49:30.864 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:49:30.865 INFO:teuthology.orchestra.run.smithi049.stdout:Total 15 MB/s | 225 MB 00:15 2024-03-26T17:49:30.886 INFO:teuthology.orchestra.run.smithi171.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-03-26T17:49:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T17:49:30.928 INFO:teuthology.orchestra.run.smithi171.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-03-26T17:49:30.969 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T17:49:30.986 INFO:teuthology.orchestra.run.smithi171.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-03-26T17:49:31.003 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T17:49:31.037 INFO:teuthology.orchestra.run.smithi171.stdout:(113/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-03-26T17:49:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T17:49:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T17:49:31.170 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T17:49:31.195 INFO:teuthology.orchestra.run.smithi171.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-03-26T17:49:31.229 INFO:teuthology.orchestra.run.smithi171.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-03-26T17:49:31.262 INFO:teuthology.orchestra.run.smithi171.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 970 kB/s | 32 kB 00:00 2024-03-26T17:49:31.329 INFO:teuthology.orchestra.run.smithi171.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-03-26T17:49:31.347 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T17:49:31.355 INFO:teuthology.orchestra.run.smithi171.stdout:(118/174): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-03-26T17:49:31.376 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T17:49:31.396 INFO:teuthology.orchestra.run.smithi171.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-03-26T17:49:31.433 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-03-26T17:49:31.454 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T17:49:31.455 INFO:teuthology.orchestra.run.smithi171.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-03-26T17:49:31.497 INFO:teuthology.orchestra.run.smithi171.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 846 kB/s | 35 kB 00:00 2024-03-26T17:49:31.522 INFO:teuthology.orchestra.run.smithi171.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-03-26T17:49:31.581 INFO:teuthology.orchestra.run.smithi171.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-26T17:49:31.640 INFO:teuthology.orchestra.run.smithi171.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-26T17:49:31.681 INFO:teuthology.orchestra.run.smithi171.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 258 kB/s | 11 kB 00:00 2024-03-26T17:49:31.723 INFO:teuthology.orchestra.run.smithi171.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 426 kB/s | 18 kB 00:00 2024-03-26T17:49:31.734 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-03-26T17:49:31.734 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-03-26T17:49:31.765 INFO:teuthology.orchestra.run.smithi171.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 558 kB/s | 23 kB 00:00 2024-03-26T17:49:31.824 INFO:teuthology.orchestra.run.smithi171.stdout:(128/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-03-26T17:49:31.849 INFO:teuthology.orchestra.run.smithi171.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-03-26T17:49:31.875 INFO:teuthology.orchestra.run.smithi171.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-03-26T17:49:31.917 INFO:teuthology.orchestra.run.smithi171.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-03-26T17:49:31.958 INFO:teuthology.orchestra.run.smithi171.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-03-26T17:49:32.009 INFO:teuthology.orchestra.run.smithi171.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-26T17:49:32.084 INFO:teuthology.orchestra.run.smithi171.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-03-26T17:49:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T17:49:32.143 INFO:teuthology.orchestra.run.smithi171.stdout:(135/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-03-26T17:49:32.168 INFO:teuthology.orchestra.run.smithi171.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-03-26T17:49:32.207 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T17:49:32.210 INFO:teuthology.orchestra.run.smithi171.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 684 kB/s | 28 kB 00:00 2024-03-26T17:49:32.235 INFO:teuthology.orchestra.run.smithi171.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-26T17:49:32.260 INFO:teuthology.orchestra.run.smithi171.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-03-26T17:49:32.302 INFO:teuthology.orchestra.run.smithi171.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-03-26T17:49:32.328 INFO:teuthology.orchestra.run.smithi171.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-03-26T17:49:32.353 INFO:teuthology.orchestra.run.smithi171.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-03-26T17:49:32.378 INFO:teuthology.orchestra.run.smithi171.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-03-26T17:49:32.420 INFO:teuthology.orchestra.run.smithi171.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-03-26T17:49:32.462 INFO:teuthology.orchestra.run.smithi171.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-26T17:49:32.546 INFO:teuthology.orchestra.run.smithi171.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-03-26T17:49:32.571 INFO:teuthology.orchestra.run.smithi171.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-03-26T17:49:32.613 INFO:teuthology.orchestra.run.smithi171.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-26T17:49:32.822 INFO:teuthology.orchestra.run.smithi171.stdout:(149/174): python3-kubernetes-26.1.0-2.el9.noar 1.1 MB/s | 1.0 MB 00:00 2024-03-26T17:49:32.905 INFO:teuthology.orchestra.run.smithi171.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2024-03-26T17:49:33.055 INFO:teuthology.orchestra.run.smithi171.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.1 MB/s | 321 kB 00:00 2024-03-26T17:49:33.214 INFO:teuthology.orchestra.run.smithi171.stdout:(152/174): librbd1-19.0.0-2311.g36051606.el9.x8 8.0 MB/s | 3.1 MB 00:00 2024-03-26T17:49:33.413 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T17:49:33.486 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T17:49:33.498 INFO:teuthology.orchestra.run.smithi171.stdout:(153/174): librados2-19.0.0-2311.g36051606.el9. 3.8 MB/s | 3.4 MB 00:00 2024-03-26T17:49:33.508 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T17:49:33.539 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T17:49:33.567 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T17:49:33.858 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-03-26T17:49:33.858 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-03-26T17:49:33.883 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T17:49:33.889 INFO:teuthology.orchestra.run.smithi171.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 107 kB/s | 42 kB 00:00 2024-03-26T17:49:33.914 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:49:33.965 INFO:teuthology.orchestra.run.smithi171.stdout:(155/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.5 MB/s | 2.3 MB 00:00 2024-03-26T17:49:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:49:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 125/198 2024-03-26T17:49:34.054 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T17:49:34.090 INFO:teuthology.orchestra.run.smithi171.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 571 kB/s | 71 kB 00:00 2024-03-26T17:49:34.132 INFO:teuthology.orchestra.run.smithi171.stdout:(157/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.0 MB/s | 1.8 MB 00:00 2024-03-26T17:49:34.165 INFO:teuthology.orchestra.run.smithi171.stdout:(158/174): NetworkManager-tui-1.47.2-1.el9.x86_ 901 kB/s | 248 kB 00:00 2024-03-26T17:49:34.190 INFO:teuthology.orchestra.run.smithi171.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 362 kB/s | 36 kB 00:00 2024-03-26T17:49:34.241 INFO:teuthology.orchestra.run.smithi171.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-03-26T17:49:34.274 INFO:teuthology.orchestra.run.smithi171.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-26T17:49:34.309 INFO:teuthology.orchestra.run.smithi171.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 196 kB/s | 23 kB 00:00 2024-03-26T17:49:34.344 INFO:teuthology.orchestra.run.smithi171.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 108 kB 00:00 2024-03-26T17:49:34.378 INFO:teuthology.orchestra.run.smithi171.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-26T17:49:34.411 INFO:teuthology.orchestra.run.smithi171.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-26T17:49:34.445 INFO:teuthology.orchestra.run.smithi171.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 632 kB/s | 64 kB 00:00 2024-03-26T17:49:34.479 INFO:teuthology.orchestra.run.smithi171.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 285 kB/s | 29 kB 00:00 2024-03-26T17:49:34.521 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T17:49:34.529 INFO:teuthology.orchestra.run.smithi171.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2024-03-26T17:49:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:49:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:49:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 129/198 2024-03-26T17:49:34.654 INFO:teuthology.orchestra.run.smithi171.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-03-26T17:49:34.757 INFO:teuthology.orchestra.run.smithi171.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.9 MB/s | 2.2 MB 00:00 2024-03-26T17:49:34.789 INFO:teuthology.orchestra.run.smithi171.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 232 kB/s | 31 kB 00:00 2024-03-26T17:49:34.881 INFO:teuthology.orchestra.run.smithi171.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.7 MB/s | 1.1 MB 00:00 2024-03-26T17:49:34.906 INFO:teuthology.orchestra.run.smithi171.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 636 kB/s | 74 kB 00:00 2024-03-26T17:49:34.956 INFO:teuthology.orchestra.run.smithi171.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.8 MB/s | 2.3 MB 00:00 2024-03-26T17:49:34.962 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:49:34.962 INFO:teuthology.orchestra.run.smithi171.stdout:Total 14 MB/s | 225 MB 00:16 2024-03-26T17:49:35.532 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-03-26T17:49:35.812 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-03-26T17:49:35.812 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-03-26T17:49:36.921 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:49:36.957 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:49:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:49:37.535 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:49:37.562 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-03-26T17:49:37.562 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-03-26T17:49:37.616 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:49:37.616 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T17:49:37.616 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T17:49:37.617 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:37.653 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:49:41.338 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-03-26T17:49:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T17:49:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T17:49:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:49:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:49:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T17:49:41.615 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T17:49:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T17:49:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T17:49:41.738 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T17:49:42.085 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T17:49:42.137 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:49:42.262 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:49:42.288 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:49:42.364 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:49:42.401 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:49:42.446 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T17:49:42.650 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T17:49:42.708 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T17:49:42.735 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:49:42.840 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:49:42.870 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:49:42.918 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:49:42.957 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T17:49:42.979 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:49:42.995 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:49:43.153 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:49:43.186 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:49:43.225 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T17:49:43.405 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T17:49:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:49:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:49:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:49:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:49:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:49:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T17:49:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T17:49:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T17:49:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T17:49:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T17:49:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T17:49:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T17:49:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T17:49:44.495 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-03-26T17:49:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T17:49:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T17:49:44.619 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T17:49:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T17:49:44.653 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T17:49:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T17:49:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T17:49:44.693 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:49:44.702 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:49:44.742 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T17:49:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T17:49:44.796 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T17:49:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 39/198 2024-03-26T17:49:44.830 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T17:49:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 40/198 2024-03-26T17:49:44.864 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T17:49:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 41/198 2024-03-26T17:49:44.906 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T17:49:44.941 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T17:49:44.993 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T17:49:45.069 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T17:49:45.207 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T17:49:45.227 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T17:49:45.271 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T17:49:45.282 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:49:45.403 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:49:45.434 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:49:45.525 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:49:45.555 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:49:45.592 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T17:49:45.770 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T17:49:45.822 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T17:49:45.828 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T17:49:45.857 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:49:45.908 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T17:49:45.960 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:49:45.983 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:49:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T17:49:46.039 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:49:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:49:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:49:46.081 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T17:49:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T17:49:46.102 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:49:46.117 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:49:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T17:49:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 53/198 2024-03-26T17:49:46.200 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T17:49:46.311 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:49:46.333 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:49:46.365 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T17:49:46.464 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T17:49:46.490 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T17:49:46.578 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:49:46.629 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:49:46.668 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T17:49:46.700 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:49:46.756 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:49:46.802 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T17:49:46.958 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:49:46.990 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T17:49:47.060 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T17:49:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T17:49:47.149 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T17:49:47.179 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T17:49:47.182 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T17:49:47.205 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T17:49:47.238 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T17:49:47.241 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T17:49:47.312 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T17:49:47.315 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T17:49:47.351 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T17:49:47.370 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T17:49:47.388 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T17:49:47.408 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T17:49:47.435 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T17:49:47.470 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T17:49:47.473 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T17:49:47.508 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T17:49:47.515 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T17:49:47.551 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T17:49:47.574 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T17:49:47.584 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T17:49:47.613 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T17:49:47.681 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T17:49:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T17:49:47.718 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 39/198 2024-03-26T17:49:47.761 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 40/198 2024-03-26T17:49:47.762 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T17:49:47.790 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 41/198 2024-03-26T17:49:47.800 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T17:49:47.842 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T17:49:47.861 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T17:49:47.905 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T17:49:47.939 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T17:49:47.980 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T17:49:47.991 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T17:49:48.023 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T17:49:48.049 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T17:49:48.073 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T17:49:48.123 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T17:49:48.153 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T17:49:48.175 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T17:49:48.287 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T17:49:48.393 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T17:49:48.432 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T17:49:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T17:49:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T17:49:48.696 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T17:49:48.728 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T17:49:48.769 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T17:49:48.780 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T17:49:48.874 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T17:49:48.911 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T17:49:48.917 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:49:48.927 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:49:48.951 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T17:49:48.966 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T17:49:48.991 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T17:49:49.007 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T17:49:49.021 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T17:49:49.045 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 53/198 2024-03-26T17:49:49.049 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T17:49:49.075 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T17:49:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T17:49:49.123 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T17:49:49.155 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T17:49:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T17:49:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T17:49:49.295 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T17:49:49.311 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T17:49:49.328 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T17:49:49.362 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T17:49:49.396 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T17:49:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:49:49.477 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-03-26T17:49:49.477 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-03-26T17:49:49.477 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-03-26T17:49:49.478 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-03-26T17:49:49.478 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-03-26T17:49:49.478 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-03-26T17:49:49.478 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-03-26T17:49:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T17:49:49.507 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T17:49:49.577 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T17:49:49.580 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T17:49:49.624 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T17:49:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T17:49:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:49:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:49:49.654 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:49:49.655 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T17:49:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T17:49:49.706 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T17:49:49.773 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T17:49:49.806 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T17:49:49.821 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T17:49:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T17:49:49.876 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T17:49:49.948 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T17:49:49.980 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T17:49:50.017 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T17:49:50.063 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T17:49:50.103 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T17:49:50.146 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T17:49:50.407 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T17:49:50.544 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T17:49:50.595 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T17:49:50.626 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T17:49:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T17:49:50.667 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T17:49:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T17:49:50.715 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T17:49:50.762 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T17:49:50.764 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T17:49:50.793 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T17:49:50.833 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T17:49:50.854 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T17:49:50.859 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T17:49:50.876 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T17:49:50.900 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T17:49:50.949 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T17:49:51.078 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T17:49:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T17:49:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T17:49:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:49:51.178 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T17:49:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:49:51.186 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:49:51.186 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T17:49:51.186 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:49:51.186 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:49:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T17:49:51.219 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T17:49:51.268 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T17:49:51.324 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T17:49:51.411 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:49:51.433 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:49:51.433 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:49:51.433 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T17:49:51.433 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:49:51.433 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:49:51.434 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:51.509 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T17:49:51.558 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T17:49:51.694 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T17:49:51.748 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T17:49:51.780 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T17:49:51.811 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T17:49:51.840 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T17:49:51.874 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T17:49:51.875 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T17:49:51.878 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:49:51.901 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:49:51.901 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:49:51.901 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T17:49:51.901 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:49:51.901 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:49:51.901 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:51.913 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T17:49:51.945 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T17:49:51.967 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T17:49:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T17:49:52.043 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T17:49:52.078 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T17:49:52.112 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T17:49:52.149 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T17:49:52.153 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T17:49:52.188 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T17:49:52.277 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T17:49:52.359 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T17:49:52.423 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:49:52.427 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T17:49:52.454 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:49:52.459 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T17:49:52.486 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 139/198 2024-03-26T17:49:52.506 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T17:49:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 140/198 2024-03-26T17:49:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:49:52.604 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T17:49:52.662 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T17:49:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:49:52.844 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:49:53.390 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T17:49:53.443 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T17:49:53.477 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T17:49:53.520 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T17:49:53.525 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T17:49:53.615 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T17:49:53.615 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T17:49:53.645 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T17:49:53.654 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T17:49:53.676 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T17:49:53.704 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T17:49:53.813 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:49:53.830 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T17:49:53.836 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:49:53.860 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T17:49:53.937 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:49:53.939 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T17:49:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T17:49:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:49:54.101 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 144/198 2024-03-26T17:49:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:49:54.142 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:49:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:49:54.153 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:49:54.153 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T17:49:54.153 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:49:54.153 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:49:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:54.172 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 125/198 2024-03-26T17:49:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:49:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:49:54.208 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T17:49:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:19.0.0-2311.g36051606.el9.x86_64 147/198 2024-03-26T17:49:54.580 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T17:49:54.685 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T17:49:54.694 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T17:49:54.720 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:49:54.791 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:49:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 129/198 2024-03-26T17:49:55.136 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T17:49:55.167 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:49:55.189 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:49:55.189 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:49:55.189 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T17:49:55.189 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:49:55.189 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:49:55.189 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:55.234 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:49:55.255 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:49:55.255 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:49:55.255 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T17:49:55.256 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:55.301 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:49:55.323 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:49:55.324 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:49:55.324 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T17:49:55.324 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:49:55.324 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:49:55.324 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:55.627 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:49:55.649 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:49:55.649 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:49:55.649 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T17:49:55.649 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:49:55.650 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:49:55.650 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:49:55.905 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T17:49:55.992 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T17:49:56.023 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T17:49:56.054 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T17:49:56.082 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T17:49:56.395 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T17:49:56.421 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:49:56.511 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:49:56.543 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 125/198 2024-03-26T17:49:56.570 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T17:49:57.031 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T17:49:57.066 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:49:57.138 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:49:57.160 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 129/198 2024-03-26T17:49:57.163 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:49:57.193 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:49:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:49:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:49:57.895 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:49:57.896 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T17:49:57.896 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T17:49:57.896 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:49:57.931 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:49:59.434 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:49:59.491 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:50:00.028 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:50:00.062 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:50:00.129 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:50:00.129 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T17:50:00.129 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T17:50:00.130 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:00.148 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:50:00.164 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:50:00.194 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 154/198 2024-03-26T17:50:00.270 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 155/198 2024-03-26T17:50:00.297 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T17:50:00.480 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 157/198 2024-03-26T17:50:00.599 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T17:50:00.636 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 159/198 2024-03-26T17:50:00.675 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T17:50:00.737 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T17:50:00.780 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T17:50:00.813 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 163/198 2024-03-26T17:50:00.853 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T17:50:00.882 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T17:50:00.937 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T17:50:00.989 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T17:50:01.023 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T17:50:01.579 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:50:01.614 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:50:01.634 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:50:02.448 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 170/198 2024-03-26T17:50:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T17:50:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T17:50:02.648 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T17:50:02.688 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T17:50:02.713 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T17:50:02.739 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T17:50:02.765 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T17:50:02.765 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:50:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:50:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T17:50:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:50:02.976 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:50:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T17:50:03.035 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T17:50:03.061 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T17:50:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T17:50:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T17:50:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T17:50:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T17:50:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T17:50:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T17:50:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:50:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T17:50:03.297 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:50:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:50:03.486 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:50:03.487 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:50:03.569 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:50:03.594 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:50:03.626 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T17:50:03.653 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T17:50:03.678 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T17:50:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:50:08.530 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:50:08.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:19.0.0-2311.g36051606.el9.x86_64 1/198 2024-03-26T17:50:08.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2/198 2024-03-26T17:50:08.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 3/198 2024-03-26T17:50:08.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 4/198 2024-03-26T17:50:08.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2311.g36051 5/198 2024-03-26T17:50:08.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 6/198 2024-03-26T17:50:08.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 7/198 2024-03-26T17:50:08.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 8/198 2024-03-26T17:50:08.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 9/198 2024-03-26T17:50:08.531 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 13/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 14/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 15/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 17/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 18/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 19/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 20/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 21/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 22/198 2024-03-26T17:50:08.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 23/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 24/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 25/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 26/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 27/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 28/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 29/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 30/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 31/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 32/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 33/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 34/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 35/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 36/198 2024-03-26T17:50:08.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 37/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:19.0.0-2311.g36051606.el9.noarch 38/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 39/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T17:50:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T17:50:08.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T17:50:08.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T17:50:08.537 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T17:50:08.538 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T17:50:08.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T17:50:08.540 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T17:50:08.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T17:50:08.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T17:50:08.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T17:50:08.544 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T17:50:08.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T17:50:09.936 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:50:09.936 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-03-26T17:50:09.936 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-03-26T17:50:09.936 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-03-26T17:50:09.936 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-03-26T17:50:09.936 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-03-26T17:50:09.936 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-03-26T17:50:09.936 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-03-26T17:50:09.936 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:50:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:50:10.117 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:10.118 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T17:50:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:11.582 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T17:50:11.583 INFO:teuthology.orchestra.run.smithi115.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T17:50:11.584 INFO:teuthology.orchestra.run.smithi115.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.585 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.586 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T17:50:11.587 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: jq-1.6-16.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T17:50:11.588 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T17:50:11.589 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T17:50:11.590 INFO:teuthology.orchestra.run.smithi115.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T17:50:11.591 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T17:50:11.592 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T17:50:11.593 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T17:50:11.594 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T17:50:11.594 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T17:50:11.594 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T17:50:11.594 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T17:50:11.594 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T17:50:11.594 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T17:50:11.594 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T17:50:11.594 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T17:50:11.594 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T17:50:11.594 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T17:50:11.595 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T17:50:11.597 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T17:50:11.597 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T17:50:11.597 INFO:teuthology.orchestra.run.smithi115.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T17:50:11.597 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T17:50:11.597 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T17:50:11.597 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T17:50:11.597 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-03-26T17:50:11.597 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-03-26T17:50:11.638 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:50:11.667 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:50:11.667 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:11.667 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T17:50:11.667 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:50:11.667 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:50:11.667 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:11.897 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:50:11.921 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:50:11.921 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:11.921 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T17:50:11.921 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:50:11.922 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:50:11.922 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:12.028 DEBUG:teuthology.parallel:result is None 2024-03-26T17:50:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:50:12.260 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-03-26T17:50:12.260 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-03-26T17:50:12.260 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-03-26T17:50:12.260 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-03-26T17:50:12.260 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-03-26T17:50:12.260 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-03-26T17:50:12.260 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-03-26T17:50:12.260 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:50:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:50:12.413 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:12.413 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T17:50:12.413 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:50:12.413 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:50:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:12.425 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:50:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:50:12.440 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:12.440 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T17:50:12.440 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T17:50:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:50:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:50:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 139/198 2024-03-26T17:50:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 140/198 2024-03-26T17:50:13.088 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:50:13.334 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:50:13.372 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:50:13.921 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:50:13.946 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:50:13.946 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:13.947 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T17:50:13.947 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:50:13.947 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:50:13.947 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:14.180 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:50:14.202 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:50:14.202 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:14.203 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T17:50:14.203 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:50:14.203 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:50:14.203 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:14.386 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:50:14.414 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:50:14.520 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:50:14.648 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:50:14.669 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:50:14.669 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:14.670 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T17:50:14.670 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:50:14.670 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:50:14.670 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:14.692 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 144/198 2024-03-26T17:50:14.724 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:50:14.748 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:50:14.748 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:14.748 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T17:50:14.748 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:50:14.749 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:50:14.749 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:14.758 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T17:50:14.800 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:50:14.811 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:50:14.872 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:19.0.0-2311.g36051606.el9.x86_64 147/198 2024-03-26T17:50:15.160 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:50:15.191 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:50:15.224 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 139/198 2024-03-26T17:50:15.283 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 140/198 2024-03-26T17:50:15.306 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:50:15.541 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:50:15.582 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:50:15.793 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T17:50:15.827 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:50:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:50:15.851 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:15.851 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T17:50:15.851 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:50:15.852 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:50:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:50:15.917 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:50:15.917 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:15.917 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T17:50:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:15.970 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:50:15.994 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:50:15.994 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:15.994 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T17:50:15.994 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:50:15.994 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:50:15.994 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:16.322 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:50:16.345 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:50:16.345 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:16.345 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T17:50:16.345 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:50:16.345 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:50:16.345 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:16.548 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:50:16.575 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:50:16.676 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:50:16.847 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 144/198 2024-03-26T17:50:16.886 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:50:16.909 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:50:16.909 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:16.909 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T17:50:16.909 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:50:16.909 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:50:16.909 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:16.953 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:50:16.964 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:50:17.010 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.0.0-2311.g36051606.el9.x86_64 147/198 2024-03-26T17:50:17.883 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T17:50:17.914 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:50:17.937 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:50:17.937 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:17.937 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T17:50:17.937 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:50:17.937 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:50:17.937 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:17.973 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:50:17.995 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:50:17.996 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:17.996 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T17:50:17.996 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:18.041 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:50:18.063 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:50:18.064 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:18.064 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T17:50:18.064 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:50:18.064 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:50:18.064 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:18.367 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:50:18.389 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:50:18.390 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:50:18.390 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T17:50:18.390 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:50:18.390 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:50:18.390 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:21.092 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:50:21.134 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 154/198 2024-03-26T17:50:21.164 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 155/198 2024-03-26T17:50:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T17:50:21.387 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 157/198 2024-03-26T17:50:21.511 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T17:50:21.556 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 159/198 2024-03-26T17:50:21.611 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T17:50:21.664 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T17:50:21.707 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T17:50:21.740 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 163/198 2024-03-26T17:50:21.780 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T17:50:21.810 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T17:50:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T17:50:21.955 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T17:50:21.992 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T17:50:22.632 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:50:22.673 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:50:22.693 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:50:22.840 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:50:22.881 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 154/198 2024-03-26T17:50:22.912 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 155/198 2024-03-26T17:50:22.939 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T17:50:23.123 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 157/198 2024-03-26T17:50:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T17:50:23.281 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 159/198 2024-03-26T17:50:23.320 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T17:50:23.373 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T17:50:23.408 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T17:50:23.447 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 163/198 2024-03-26T17:50:23.482 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T17:50:23.520 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T17:50:23.566 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T17:50:23.570 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 170/198 2024-03-26T17:50:23.628 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T17:50:23.659 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T17:50:23.661 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T17:50:23.701 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T17:50:23.798 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T17:50:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T17:50:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T17:50:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T17:50:23.906 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T17:50:23.907 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:50:23.995 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:50:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T17:50:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:50:24.108 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:50:24.143 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T17:50:24.176 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T17:50:24.200 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:50:24.201 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T17:50:24.226 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T17:50:24.244 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:50:24.261 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T17:50:24.264 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:50:24.308 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T17:50:24.348 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T17:50:24.373 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T17:50:24.398 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T17:50:24.424 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:50:24.449 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T17:50:24.455 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:50:24.455 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:50:24.643 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:50:24.644 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:50:24.726 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:50:24.751 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:50:24.784 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T17:50:24.810 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T17:50:24.835 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T17:50:24.905 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:50:25.097 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 170/198 2024-03-26T17:50:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T17:50:25.234 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T17:50:25.316 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T17:50:25.356 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T17:50:25.381 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T17:50:25.407 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T17:50:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T17:50:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:50:25.522 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:50:25.551 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T17:50:25.552 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:50:25.637 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:50:25.663 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T17:50:25.696 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T17:50:25.722 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T17:50:25.747 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T17:50:25.780 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T17:50:25.821 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T17:50:25.853 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T17:50:25.879 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T17:50:25.904 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T17:50:25.929 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:50:25.955 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T17:50:25.962 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:50:25.962 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:50:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:50:26.150 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:50:26.216 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:50:26.241 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:50:26.273 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T17:50:26.301 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T17:50:26.326 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T17:50:26.390 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:50:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:50:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:19.0.0-2311.g36051606.el9.x86_64 1/198 2024-03-26T17:50:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2/198 2024-03-26T17:50:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 3/198 2024-03-26T17:50:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 4/198 2024-03-26T17:50:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2311.g36051 5/198 2024-03-26T17:50:29.890 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 6/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 7/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 8/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 9/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 13/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 14/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 15/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 17/198 2024-03-26T17:50:29.891 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 18/198 2024-03-26T17:50:29.892 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 19/198 2024-03-26T17:50:29.892 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 20/198 2024-03-26T17:50:29.892 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 21/198 2024-03-26T17:50:29.892 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 22/198 2024-03-26T17:50:29.892 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 23/198 2024-03-26T17:50:29.892 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 24/198 2024-03-26T17:50:29.892 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 25/198 2024-03-26T17:50:29.892 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 26/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 27/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 28/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 29/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 30/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 31/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 32/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 33/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 34/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 35/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 36/198 2024-03-26T17:50:29.909 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 37/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:19.0.0-2311.g36051606.el9.noarch 38/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 39/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T17:50:29.910 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T17:50:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T17:50:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T17:50:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T17:50:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T17:50:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T17:50:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T17:50:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T17:50:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T17:50:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T17:50:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T17:50:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T17:50:31.500 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.0.0-2311.g36051606.el9.x86_64 1/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 3/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 4/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2311.g36051 5/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 6/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 7/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 8/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 9/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:50:31.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 13/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 14/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 15/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 17/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 18/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 19/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 20/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 21/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 22/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 23/198 2024-03-26T17:50:31.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 24/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 25/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 26/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 27/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 28/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 29/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 30/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 31/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 32/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 33/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 34/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 35/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 36/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 37/198 2024-03-26T17:50:31.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.0.0-2311.g36051606.el9.noarch 38/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 39/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T17:50:31.504 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T17:50:31.505 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T17:50:31.506 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T17:50:31.507 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T17:50:31.508 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T17:50:31.509 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T17:50:31.510 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T17:50:31.511 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T17:50:31.512 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T17:50:31.513 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T17:50:31.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T17:50:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T17:50:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T17:50:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T17:50:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T17:50:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T17:50:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T17:50:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T17:50:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T17:50:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:50:31.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T17:50:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T17:50:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T17:50:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:50:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T17:50:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T17:50:31.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T17:50:33.069 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T17:50:33.069 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:33.069 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-03-26T17:50:33.069 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:33.072 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T17:50:33.073 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: jq-1.6-16.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.074 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T17:50:33.083 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T17:50:33.083 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:50:33.083 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-03-26T17:50:33.551 DEBUG:teuthology.parallel:result is None 2024-03-26T17:50:34.637 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T17:50:34.638 INFO:teuthology.orchestra.run.smithi171.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T17:50:34.639 INFO:teuthology.orchestra.run.smithi171.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.640 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.641 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T17:50:34.642 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: jq-1.6-16.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T17:50:34.643 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T17:50:34.646 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T17:50:34.647 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T17:50:34.647 INFO:teuthology.orchestra.run.smithi171.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T17:50:34.647 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T17:50:34.647 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T17:50:34.647 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T17:50:34.647 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T17:50:34.647 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T17:50:34.647 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T17:50:34.647 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T17:50:34.647 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T17:50:34.648 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T17:50:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T17:50:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T17:50:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T17:50:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T17:50:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T17:50:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T17:50:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T17:50:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T17:50:34.649 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T17:50:34.650 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T17:50:34.651 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T17:50:34.652 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:50:34.653 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T17:50:34.654 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T17:50:34.654 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T17:50:34.654 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T17:50:34.654 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T17:50:34.654 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T17:50:34.654 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T17:50:34.654 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:50:34.654 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-03-26T17:50:35.159 DEBUG:teuthology.parallel:result is None 2024-03-26T17:50:35.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:50:35.425 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T17:50:35.450 INFO:teuthology.orchestra.run.smithi049.stdout:19.0.0-2311.g36051606.el9 2024-03-26T17:50:35.451 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2311.g36051606.el9 2024-03-26T17:50:35.451 INFO:teuthology.task.install:The correct ceph version 19.0.0-2311.g36051606 is installed. 2024-03-26T17:50:35.453 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:50:35.777 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T17:50:35.803 INFO:teuthology.orchestra.run.smithi115.stdout:19.0.0-2311.g36051606.el9 2024-03-26T17:50:35.803 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2311.g36051606.el9 2024-03-26T17:50:35.803 INFO:teuthology.task.install:The correct ceph version 19.0.0-2311.g36051606 is installed. 2024-03-26T17:50:35.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:50:35.939 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T17:50:35.965 INFO:teuthology.orchestra.run.smithi171.stdout:19.0.0-2311.g36051606.el9 2024-03-26T17:50:35.965 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2311.g36051606.el9 2024-03-26T17:50:35.965 INFO:teuthology.task.install:The correct ceph version 19.0.0-2311.g36051606 is installed. 2024-03-26T17:50:35.967 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-26T17:50:35.967 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:50:35.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T17:50:36.006 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:50:36.007 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T17:50:36.041 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:50:36.041 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T17:50:36.076 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-26T17:50:36.077 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:50:36.077 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T17:50:36.112 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T17:50:36.187 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:50:36.187 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T17:50:36.219 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T17:50:36.291 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:50:36.291 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T17:50:36.323 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T17:50:36.394 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-26T17:50:36.395 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:50:36.395 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T17:50:36.431 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T17:50:36.507 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:50:36.507 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T17:50:36.535 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T17:50:36.602 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:50:36.602 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T17:50:36.634 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T17:50:36.704 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-26T17:50:36.705 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:50:36.705 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T17:50:36.741 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T17:50:36.814 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:50:36.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T17:50:36.842 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T17:50:36.909 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:50:36.909 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T17:50:36.939 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T17:50:37.008 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-26T17:50:37.108 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', '\\(PG_DEGRADED', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-26T17:50:37.108 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:50:37.109 INFO:tasks.cephadm:Cluster fsid is 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:50:37.109 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-26T17:50:37.109 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.49', 'mon.b': '172.21.15.115', 'mon.c': '172.21.15.171'} 2024-03-26T17:50:37.109 INFO:tasks.cephadm:First mon is mon.a on smithi049 2024-03-26T17:50:37.109 INFO:tasks.cephadm:First mgr is x 2024-03-26T17:50:37.109 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-26T17:50:37.109 DEBUG:teuthology.orchestra.run.smithi049:> sudo hostname $(hostname -s) 2024-03-26T17:50:37.145 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2024-03-26T17:50:37.172 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2024-03-26T17:50:37.203 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-26T17:50:37.203 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:50:37.414 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77808/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2311-g36051606', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2311.g36051606'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-20 17:53:08.349574', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240320.145742', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-26T17:50:37.561 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-batrick-testing-20240320.145742, sha1 360516069d9393362c4cc6eb9371680fe16d66ab from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:50:37.563 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm 2024-03-26T17:50:37.563 DEBUG:teuthology.orchestra.run.smithi049:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T17:50:38.201 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-r--r--. 1 ubuntu ubuntu 769283 Mar 26 17:50 /home/ubuntu/cephtest/cephadm 2024-03-26T17:50:38.201 DEBUG:teuthology.orchestra.run.smithi115:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T17:50:38.819 INFO:teuthology.orchestra.run.smithi115.stdout:-rw-r--r--. 1 ubuntu ubuntu 769283 Mar 26 17:50 /home/ubuntu/cephtest/cephadm 2024-03-26T17:50:38.820 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T17:50:39.432 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-r--r--. 1 ubuntu ubuntu 769283 Mar 26 17:50 /home/ubuntu/cephtest/cephadm 2024-03-26T17:50:39.433 DEBUG:teuthology.orchestra.run.smithi049:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T17:50:39.452 DEBUG:teuthology.orchestra.run.smithi115:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T17:50:39.471 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T17:50:39.506 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab on all hosts... 2024-03-26T17:50:39.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab pull 2024-03-26T17:50:39.509 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab pull 2024-03-26T17:50:39.514 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab pull 2024-03-26T17:50:40.346 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T17:50:40.439 INFO:teuthology.orchestra.run.smithi049.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T17:50:40.451 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T17:51:07.237 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-26T17:51:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T17:51:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016", 2024-03-26T17:51:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2024-03-26T17:51:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9" 2024-03-26T17:51:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-03-26T17:51:07.238 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-26T17:51:30.514 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-03-26T17:51:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T17:51:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016", 2024-03-26T17:51:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2024-03-26T17:51:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9" 2024-03-26T17:51:30.514 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-03-26T17:51:30.514 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-03-26T17:51:33.405 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T17:51:33.405 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T17:51:33.406 INFO:teuthology.orchestra.run.smithi049.stdout: "image_id": "497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016", 2024-03-26T17:51:33.406 INFO:teuthology.orchestra.run.smithi049.stdout: "repo_digests": [ 2024-03-26T17:51:33.406 INFO:teuthology.orchestra.run.smithi049.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9" 2024-03-26T17:51:33.406 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T17:51:33.406 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T17:51:33.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph 2024-03-26T17:51:33.477 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2024-03-26T17:51:33.513 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2024-03-26T17:51:33.550 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /etc/ceph 2024-03-26T17:51:33.581 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2024-03-26T17:51:33.613 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2024-03-26T17:51:33.644 INFO:tasks.cephadm:Writing seed config... 2024-03-26T17:51:33.645 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-26T17:51:33.645 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-26T17:51:33.645 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-26T17:51:33.645 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-26T17:51:33.645 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-26T17:51:33.645 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-26T17:51:33.645 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-26T17:51:33.646 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-26T17:51:33.647 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-26T17:51:33.648 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-26T17:51:33.649 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:51:33.649 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-26T17:51:33.665 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5a3632a4-eb99-11ee-95d0-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-26T17:51:33.665 DEBUG:teuthology.orchestra.run.smithi049:mon.a> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.a.service 2024-03-26T17:51:33.707 DEBUG:teuthology.orchestra.run.smithi049:mgr.x> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.x.service 2024-03-26T17:51:33.749 INFO:tasks.cephadm:Bootstrapping... 2024-03-26T17:51:33.750 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab -v bootstrap --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.49 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-26T17:51:33.950 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:51:33.951 INFO:teuthology.orchestra.run.smithi049.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab', '-v', 'bootstrap', '--fsid', '5a3632a4-eb99-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.49', '--skip-admin-label'] 2024-03-26T17:51:33.951 INFO:teuthology.orchestra.run.smithi049.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-26T17:51:33.951 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying podman|docker is present... 2024-03-26T17:51:33.977 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T17:51:33.977 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying lvm2 is present... 2024-03-26T17:51:33.977 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying time synchronization is in place... 2024-03-26T17:51:33.985 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T17:51:33.985 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T17:51:33.991 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T17:51:33.991 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-26T17:51:33.998 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-03-26T17:51:34.006 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-03-26T17:51:34.006 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-03-26T17:51:34.006 INFO:teuthology.orchestra.run.smithi049.stdout:Repeating the final host check... 2024-03-26T17:51:34.028 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T17:51:34.028 INFO:teuthology.orchestra.run.smithi049.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-26T17:51:34.028 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl is present 2024-03-26T17:51:34.028 INFO:teuthology.orchestra.run.smithi049.stdout:lvcreate is present 2024-03-26T17:51:34.034 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T17:51:34.034 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T17:51:34.040 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T17:51:34.040 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-26T17:51:34.047 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2024-03-26T17:51:34.053 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2024-03-26T17:51:34.053 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2024-03-26T17:51:34.053 INFO:teuthology.orchestra.run.smithi049.stdout:Host looks OK 2024-03-26T17:51:34.053 INFO:teuthology.orchestra.run.smithi049.stdout:Cluster fsid: 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:51:34.053 INFO:teuthology.orchestra.run.smithi049.stdout:Acquiring lock 140403422671824 on /run/cephadm/5a3632a4-eb99-11ee-95d0-87774f69a715.lock 2024-03-26T17:51:34.053 INFO:teuthology.orchestra.run.smithi049.stdout:Lock 140403422671824 acquired on /run/cephadm/5a3632a4-eb99-11ee-95d0-87774f69a715.lock 2024-03-26T17:51:34.054 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 3300 ... 2024-03-26T17:51:34.054 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 6789 ... 2024-03-26T17:51:34.054 INFO:teuthology.orchestra.run.smithi049.stdout:Base mon IP(s) is [172.21.15.49:3300, 172.21.15.49:6789], mon addrv is [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-03-26T17:51:34.058 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.49 metric 100 2024-03-26T17:51:34.058 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.49 metric 100 2024-03-26T17:51:34.060 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-26T17:51:34.060 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-26T17:51:34.060 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-26T17:51:34.063 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-26T17:51:34.063 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-26T17:51:34.063 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T17:51:34.063 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-26T17:51:34.063 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-03-26T17:51:34.063 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T17:51:34.064 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-03-26T17:51:34.064 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2024-03-26T17:51:34.064 INFO:teuthology.orchestra.run.smithi049.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-26T17:51:34.064 INFO:teuthology.orchestra.run.smithi049.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-26T17:51:34.065 INFO:teuthology.orchestra.run.smithi049.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T17:51:34.525 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 2024-03-26T17:51:34.525 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T17:51:34.525 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Getting image source signatures 2024-03-26T17:51:34.525 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:0388617eb36698e8cd0b89da9480e32fffb3f5cb74210feb5d82e6789f473b79 2024-03-26T17:51:34.525 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-26T17:51:34.525 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying config sha256:497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 2024-03-26T17:51:34.525 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-26T17:51:35.345 INFO:teuthology.orchestra.run.smithi049.stdout:ceph: stdout ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev) 2024-03-26T17:51:35.345 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph version: ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev) 2024-03-26T17:51:35.345 INFO:teuthology.orchestra.run.smithi049.stdout:Extracting ceph user uid/gid from container image... 2024-03-26T17:51:36.326 INFO:teuthology.orchestra.run.smithi049.stdout:stat: stdout 167 167 2024-03-26T17:51:36.326 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial keys... 2024-03-26T17:51:37.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQAoCwNm9VjxIxAAYWTp2FJqisTi453mb8lqLw== 2024-03-26T17:51:37.995 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQApCwNmdfbqIBAA5lQFM5M7Z9dRAcrp2Ck1jg== 2024-03-26T17:51:38.960 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQAqCwNm9NXIFBAArh4lxnMXqc4+5mviF9h5qw== 2024-03-26T17:51:38.961 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial monmap... 2024-03-26T17:51:39.883 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T17:51:39.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-26T17:51:39.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:51:39.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T17:51:39.884 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool for a [v2:172.21.15.49:3300,v1:172.21.15.49:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T17:51:39.884 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = quincy 2024-03-26T17:51:39.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: set fsid to 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:51:39.884 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T17:51:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:51:39.884 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mon... 2024-03-26T17:51:40.847 INFO:teuthology.orchestra.run.smithi049.stdout:create mon.a on 2024-03-26T17:51:41.055 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-26T17:51:41.214 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-26T17:51:41.411 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5a3632a4-eb99-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-5a3632a4-eb99-11ee-95d0-87774f69a715.target. 2024-03-26T17:51:41.411 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5a3632a4-eb99-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-5a3632a4-eb99-11ee-95d0-87774f69a715.target. 2024-03-26T17:51:41.664 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.a 2024-03-26T17:51:41.664 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-26T17:51:41.845 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5a3632a4-eb99-11ee-95d0-87774f69a715.target.wants/ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@.service. 2024-03-26T17:51:41.897 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:41 smithi049 systemd[1]: Starting Ceph mon.a for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:51:42.189 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:42 smithi049 podman[31938]: 2024-03-26 17:51:42.097179176 +0000 UTC m=+0.115075045 container create f70eef60af800fae8e77e7d91855b563e0c223646b4a2c1c0c9c47bda17f5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-26T17:51:42.189 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:42 smithi049 podman[31938]: 2024-03-26 17:51:42.002474755 +0000 UTC m=+0.020370637 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:51:42.354 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T17:51:42.354 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-26T17:51:42.362 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T17:51:42.363 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-26T17:51:42.363 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-26T17:51:42.363 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T17:51:42.363 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon to start... 2024-03-26T17:51:42.363 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon... 2024-03-26T17:51:42.484 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:42 smithi049 podman[31938]: 2024-03-26 17:51:42.282738343 +0000 UTC m=+0.300634198 container init f70eef60af800fae8e77e7d91855b563e0c223646b4a2c1c0c9c47bda17f5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T17:51:42.484 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:42 smithi049 podman[31938]: 2024-03-26 17:51:42.286201112 +0000 UTC m=+0.304096968 container start f70eef60af800fae8e77e7d91855b563e0c223646b4a2c1c0c9c47bda17f5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-26T17:51:42.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:42 smithi049 bash[31938]: f70eef60af800fae8e77e7d91855b563e0c223646b4a2c1c0c9c47bda17f5a2f 2024-03-26T17:51:42.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:42 smithi049 systemd[1]: Started Ceph mon.a for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:51:42.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:42 smithi049 ceph-mon[31957]: mkfs 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:51:42.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:42 smithi049 ceph-mon[31957]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T17:51:43.480 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T17:51:43.480 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: monmap epoch 1 2024-03-26T17:51:43.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:51:43.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: last_changed 2024-03-26T17:51:39.354207+0000 2024-03-26T17:51:43.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: created 2024-03-26T17:51:39.354207+0000 2024-03-26T17:51:43.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: min_mon_release 19 (squid) 2024-03-26T17:51:43.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: election_strategy: 1 2024-03-26T17:51:43.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-26T17:51:43.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: fsmap 2024-03-26T17:51:43.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T17:51:43.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: mgrmap e1: no daemons active 2024-03-26T17:51:43.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:43 smithi049 ceph-mon[31957]: from='client.? 172.21.15.49:0/1973636552' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-26T17:51:43.484 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout cluster: 2024-03-26T17:51:43.490 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout id: 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:51:43.490 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-26T17:51:43.490 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:51:43.490 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout services: 2024-03-26T17:51:43.490 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.544201s) 2024-03-26T17:51:43.490 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-26T17:51:43.490 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout data: 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pgs: 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.818+0000 7fdbc4c56700 1 Processor -- start 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.818+0000 7fdbc4c56700 1 -- start start 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.818+0000 7fdbc4c56700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc010ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.818+0000 7fdbc4c56700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc007bda0 con 0x7fdbc01089f0 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.819+0000 7fdbbe59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc010ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.819+0000 7fdbbe59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc010ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58790/0 (socket says 172.21.15.49:58790) 2024-03-26T17:51:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.819+0000 7fdbbe59c700 1 -- 172.21.15.49:0/1744005145 learned_addr learned my addr 172.21.15.49:0/1744005145 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.819+0000 7fdbbe59c700 1 -- 172.21.15.49:0/1744005145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc010b320 con 0x7fdbc01089f0 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.819+0000 7fdbbe59c700 1 --2- 172.21.15.49:0/1744005145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc010ade0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009e80 tx=0x7fdbb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30faa3d06a7d59e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.819+0000 7fdbbd59a700 1 -- 172.21.15.49:0/1744005145 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbb0017070 con 0x7fdbc01089f0 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.820+0000 7fdbbd59a700 1 -- 172.21.15.49:0/1744005145 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb0003a20 con 0x7fdbc01089f0 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.820+0000 7fdbbd59a700 1 -- 172.21.15.49:0/1744005145 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbb0003d20 con 0x7fdbc01089f0 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.820+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1744005145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 msgr2=0x7fdbc010ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.820+0000 7fdbc4c56700 1 --2- 172.21.15.49:0/1744005145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc010ade0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009e80 tx=0x7fdbb000b330 comp rx=0 tx=0).stop 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.820+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1744005145 shutdown_connections 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.820+0000 7fdbc4c56700 1 --2- 172.21.15.49:0/1744005145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc010ade0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.820+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1744005145 >> 172.21.15.49:0/1744005145 conn(0x7fdbc0102420 msgr2=0x7fdbc0104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.820+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1744005145 shutdown_connections 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.820+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1744005145 wait complete. 2024-03-26T17:51:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.821+0000 7fdbc4c56700 1 Processor -- start 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.821+0000 7fdbc4c56700 1 -- start start 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.821+0000 7fdbc4c56700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.821+0000 7fdbc4c56700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc010b790 con 0x7fdbc01089f0 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.821+0000 7fdbbe59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.821+0000 7fdbbe59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc019b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58794/0 (socket says 172.21.15.49:58794) 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.821+0000 7fdbbe59c700 1 -- 172.21.15.49:0/1973636552 learned_addr learned my addr 172.21.15.49:0/1973636552 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.822+0000 7fdbbe59c700 1 -- 172.21.15.49:0/1973636552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc019baf0 con 0x7fdbc01089f0 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.822+0000 7fdbbe59c700 1 --2- 172.21.15.49:0/1973636552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc019b5b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdbc007b1e0 tx=0x7fdbb0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.822+0000 7fdbb77fe700 1 -- 172.21.15.49:0/1973636552 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbb0022020 con 0x7fdbc01089f0 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.822+0000 7fdbb77fe700 1 -- 172.21.15.49:0/1973636552 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb0018040 con 0x7fdbc01089f0 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.822+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1973636552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc019bd80 con 0x7fdbc01089f0 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.822+0000 7fdbb77fe700 1 -- 172.21.15.49:0/1973636552 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbb0017070 con 0x7fdbc01089f0 2024-03-26T17:51:43.493 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.822+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1973636552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc019ea70 con 0x7fdbc01089f0 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.822+0000 7fdbb77fe700 1 -- 172.21.15.49:0/1973636552 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdbb0025020 con 0x7fdbc01089f0 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.822+0000 7fdbb77fe700 1 -- 172.21.15.49:0/1973636552 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdbb000f050 con 0x7fdbc01089f0 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.823+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1973636552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc00694b0 con 0x7fdbc01089f0 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.825+0000 7fdbb77fe700 1 -- 172.21.15.49:0/1973636552 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdbb002c030 con 0x7fdbc01089f0 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.897+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1973636552 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fdbc002dd00 con 0x7fdbc01089f0 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.897+0000 7fdbb77fe700 1 -- 172.21.15.49:0/1973636552 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fdbb001e480 con 0x7fdbc01089f0 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.898+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1973636552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 msgr2=0x7fdbc019b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.898+0000 7fdbc4c56700 1 --2- 172.21.15.49:0/1973636552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc019b5b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdbc007b1e0 tx=0x7fdbb0004ad0 comp rx=0 tx=0).stop 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.898+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1973636552 shutdown_connections 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.898+0000 7fdbc4c56700 1 --2- 172.21.15.49:0/1973636552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbc01089f0 0x7fdbc019b5b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.898+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1973636552 >> 172.21.15.49:0/1973636552 conn(0x7fdbc0102420 msgr2=0x7fdbc01046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.898+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1973636552 shutdown_connections 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:42.898+0000 7fdbc4c56700 1 -- 172.21.15.49:0/1973636552 wait complete. 2024-03-26T17:51:43.494 INFO:teuthology.orchestra.run.smithi049.stdout:mon is available 2024-03-26T17:51:43.495 INFO:teuthology.orchestra.run.smithi049.stdout:Assimilating anything we can from ceph.conf... 2024-03-26T17:51:44.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:44 smithi049 ceph-mon[31957]: from='client.? 172.21.15.49:0/1728512283' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T17:51:44.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:44 smithi049 ceph-mon[31957]: from='client.? 172.21.15.49:0/1728512283' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T17:51:44.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.923+0000 7f32c6a58700 1 Processor -- start 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.923+0000 7f32c6a58700 1 -- start start 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.924+0000 7f32c6a58700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c0108310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.924+0000 7f32c6a58700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0108850 con 0x7f32c0107ef0 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.924+0000 7f32bffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c0108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:44.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.924+0000 7f32bffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c0108310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58800/0 (socket says 172.21.15.49:58800) 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.924+0000 7f32bffff700 1 -- 172.21.15.49:0/2942524300 learned_addr learned my addr 172.21.15.49:0/2942524300 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.924+0000 7f32bffff700 1 -- 172.21.15.49:0/2942524300 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c0109090 con 0x7f32c0107ef0 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.924+0000 7f32bffff700 1 --2- 172.21.15.49:0/2942524300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c0108310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f32b00098a0 tx=0x7f32b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35d0e0b571864f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.925+0000 7f32beffd700 1 -- 172.21.15.49:0/2942524300 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b0004160 con 0x7f32c0107ef0 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.925+0000 7f32beffd700 1 -- 172.21.15.49:0/2942524300 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b0004300 con 0x7f32c0107ef0 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.925+0000 7f32beffd700 1 -- 172.21.15.49:0/2942524300 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b00036a0 con 0x7f32c0107ef0 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.925+0000 7f32c6a58700 1 -- 172.21.15.49:0/2942524300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 msgr2=0x7f32c0108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.925+0000 7f32c6a58700 1 --2- 172.21.15.49:0/2942524300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c0108310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f32b00098a0 tx=0x7f32b0009bb0 comp rx=0 tx=0).stop 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.925+0000 7f32c6a58700 1 -- 172.21.15.49:0/2942524300 shutdown_connections 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.925+0000 7f32c6a58700 1 --2- 172.21.15.49:0/2942524300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c0108310 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.925+0000 7f32c6a58700 1 -- 172.21.15.49:0/2942524300 >> 172.21.15.49:0/2942524300 conn(0x7f32c0103500 msgr2=0x7f32c0105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:44.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.925+0000 7f32c6a58700 1 -- 172.21.15.49:0/2942524300 shutdown_connections 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.925+0000 7f32c6a58700 1 -- 172.21.15.49:0/2942524300 wait complete. 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.926+0000 7f32c6a58700 1 Processor -- start 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.926+0000 7f32c6a58700 1 -- start start 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.926+0000 7f32c6a58700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.926+0000 7f32c6a58700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0109560 con 0x7f32c0107ef0 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.926+0000 7f32bffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.926+0000 7f32bffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c019b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58806/0 (socket says 172.21.15.49:58806) 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.926+0000 7f32bffff700 1 -- 172.21.15.49:0/1728512283 learned_addr learned my addr 172.21.15.49:0/1728512283 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.927+0000 7f32bffff700 1 -- 172.21.15.49:0/1728512283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c019bae0 con 0x7f32c0107ef0 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.927+0000 7f32bffff700 1 --2- 172.21.15.49:0/1728512283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c019b5a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000c00 tx=0x7f32b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.927+0000 7f32bd7fa700 1 -- 172.21.15.49:0/1728512283 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b0003db0 con 0x7f32c0107ef0 2024-03-26T17:51:44.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.927+0000 7f32bd7fa700 1 -- 172.21.15.49:0/1728512283 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b0011630 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.927+0000 7f32bd7fa700 1 -- 172.21.15.49:0/1728512283 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b0011950 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.927+0000 7f32c6a58700 1 -- 172.21.15.49:0/1728512283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c019bd70 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.927+0000 7f32c6a58700 1 -- 172.21.15.49:0/1728512283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c019ea60 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.927+0000 7f32bd7fa700 1 -- 172.21.15.49:0/1728512283 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f32b001a460 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.927+0000 7f32bd7fa700 1 -- 172.21.15.49:0/1728512283 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f32b001b780 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.928+0000 7f32c6a58700 1 -- 172.21.15.49:0/1728512283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c00694b0 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:43.930+0000 7f32bd7fa700 1 -- 172.21.15.49:0/1728512283 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f32b0031020 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:44.001+0000 7f32c6a58700 1 -- 172.21.15.49:0/1728512283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f32c019c250 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:44.004+0000 7f32bd7fa700 1 -- 172.21.15.49:0/1728512283 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f32c019c250 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:44.005+0000 7f32bd7fa700 1 -- 172.21.15.49:0/1728512283 <== mon.0 v2:172.21.15.49:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f32b001a8e0 con 0x7f32c0107ef0 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:44.006+0000 7f32c6a58700 1 -- 172.21.15.49:0/1728512283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 msgr2=0x7f32c019b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:44.006+0000 7f32c6a58700 1 --2- 172.21.15.49:0/1728512283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c019b5a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000c00 tx=0x7f32b0004750 comp rx=0 tx=0).stop 2024-03-26T17:51:44.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:44.006+0000 7f32c6a58700 1 -- 172.21.15.49:0/1728512283 shutdown_connections 2024-03-26T17:51:44.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:44.006+0000 7f32c6a58700 1 --2- 172.21.15.49:0/1728512283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32c0107ef0 0x7f32c019b5a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:44.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:44.006+0000 7f32c6a58700 1 -- 172.21.15.49:0/1728512283 >> 172.21.15.49:0/1728512283 conn(0x7f32c0103500 msgr2=0x7f32c0192140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:44.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:44.006+0000 7f32c6a58700 1 -- 172.21.15.49:0/1728512283 shutdown_connections 2024-03-26T17:51:44.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:44.006+0000 7f32c6a58700 1 -- 172.21.15.49:0/1728512283 wait complete. 2024-03-26T17:51:44.660 INFO:teuthology.orchestra.run.smithi049.stdout:Generating new minimal ceph.conf... 2024-03-26T17:51:45.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:45 smithi049 ceph-mon[31957]: from='client.? 172.21.15.49:0/2818840144' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:51:45.861 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.071+0000 7f5f2c01d700 1 Processor -- start 2024-03-26T17:51:45.862 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.071+0000 7f5f2c01d700 1 -- start start 2024-03-26T17:51:45.862 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.071+0000 7f5f2c01d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2407cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:45.862 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.071+0000 7f5f2c01d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2407d020 con 0x7f5f2407e5e0 2024-03-26T17:51:45.862 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.071+0000 7f5f29db9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2407cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:45.862 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.071+0000 7f5f29db9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2407cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58812/0 (socket says 172.21.15.49:58812) 2024-03-26T17:51:45.862 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.071+0000 7f5f29db9700 1 -- 172.21.15.49:0/1876983429 learned_addr learned my addr 172.21.15.49:0/1876983429 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:45.862 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.072+0000 7f5f29db9700 1 -- 172.21.15.49:0/1876983429 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2407d1a0 con 0x7f5f2407e5e0 2024-03-26T17:51:45.865 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.072+0000 7f5f29db9700 1 --2- 172.21.15.49:0/1876983429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2407cae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5f200098a0 tx=0x7f5f20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6269987a7c416f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.072+0000 7f5f28db7700 1 -- 172.21.15.49:0/1876983429 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f20004160 con 0x7f5f2407e5e0 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.072+0000 7f5f28db7700 1 -- 172.21.15.49:0/1876983429 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5f20004300 con 0x7f5f2407e5e0 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.072+0000 7f5f28db7700 1 -- 172.21.15.49:0/1876983429 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f20011b50 con 0x7f5f2407e5e0 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.073+0000 7f5f2c01d700 1 -- 172.21.15.49:0/1876983429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 msgr2=0x7f5f2407cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.073+0000 7f5f2c01d700 1 --2- 172.21.15.49:0/1876983429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2407cae0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5f200098a0 tx=0x7f5f20009bb0 comp rx=0 tx=0).stop 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.073+0000 7f5f2c01d700 1 -- 172.21.15.49:0/1876983429 shutdown_connections 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.073+0000 7f5f2c01d700 1 --2- 172.21.15.49:0/1876983429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2407cae0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.073+0000 7f5f2c01d700 1 -- 172.21.15.49:0/1876983429 >> 172.21.15.49:0/1876983429 conn(0x7f5f24103570 msgr2=0x7f5f241059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.073+0000 7f5f2c01d700 1 -- 172.21.15.49:0/1876983429 shutdown_connections 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.073+0000 7f5f2c01d700 1 -- 172.21.15.49:0/1876983429 wait complete. 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.073+0000 7f5f2c01d700 1 Processor -- start 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.073+0000 7f5f2c01d700 1 -- start start 2024-03-26T17:51:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f2c01d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2419e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f2c01d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2410e560 con 0x7f5f2407e5e0 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f29db9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2419e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f29db9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2419e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58814/0 (socket says 172.21.15.49:58814) 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f29db9700 1 -- 172.21.15.49:0/2818840144 learned_addr learned my addr 172.21.15.49:0/2818840144 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f29db9700 1 -- 172.21.15.49:0/2818840144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2419e690 con 0x7f5f2407e5e0 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f29db9700 1 --2- 172.21.15.49:0/2818840144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2419e150 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009fb0 tx=0x7f5f20011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f1affd700 1 -- 172.21.15.49:0/2818840144 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f200038b0 con 0x7f5f2407e5e0 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f1affd700 1 -- 172.21.15.49:0/2818840144 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5f20011e50 con 0x7f5f2407e5e0 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f2c01d700 1 -- 172.21.15.49:0/2818840144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2419e920 con 0x7f5f2407e5e0 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f1affd700 1 -- 172.21.15.49:0/2818840144 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f20019420 con 0x7f5f2407e5e0 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.074+0000 7f5f2c01d700 1 -- 172.21.15.49:0/2818840144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f241a1610 con 0x7f5f2407e5e0 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.075+0000 7f5f1affd700 1 -- 172.21.15.49:0/2818840144 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5f20019ad0 con 0x7f5f2407e5e0 2024-03-26T17:51:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.075+0000 7f5f1affd700 1 -- 172.21.15.49:0/2818840144 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f20024980 con 0x7f5f2407e5e0 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.075+0000 7f5f2c01d700 1 -- 172.21.15.49:0/2818840144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f240694b0 con 0x7f5f2407e5e0 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.078+0000 7f5f1affd700 1 -- 172.21.15.49:0/2818840144 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5f20043dd0 con 0x7f5f2407e5e0 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.149+0000 7f5f2c01d700 1 -- 172.21.15.49:0/2818840144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f5f241a1ad0 con 0x7f5f2407e5e0 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.149+0000 7f5f1affd700 1 -- 172.21.15.49:0/2818840144 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f5f20032020 con 0x7f5f2407e5e0 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.150+0000 7f5f2c01d700 1 -- 172.21.15.49:0/2818840144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 msgr2=0x7f5f2419e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.150+0000 7f5f2c01d700 1 --2- 172.21.15.49:0/2818840144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2419e150 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009fb0 tx=0x7f5f20011bf0 comp rx=0 tx=0).stop 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.150+0000 7f5f2c01d700 1 -- 172.21.15.49:0/2818840144 shutdown_connections 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.150+0000 7f5f2c01d700 1 --2- 172.21.15.49:0/2818840144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5f2407e5e0 0x7f5f2419e150 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.150+0000 7f5f2c01d700 1 -- 172.21.15.49:0/2818840144 >> 172.21.15.49:0/2818840144 conn(0x7f5f24103570 msgr2=0x7f5f24104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.150+0000 7f5f2c01d700 1 -- 172.21.15.49:0/2818840144 shutdown_connections 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:45.150+0000 7f5f2c01d700 1 -- 172.21.15.49:0/2818840144 wait complete. 2024-03-26T17:51:45.868 INFO:teuthology.orchestra.run.smithi049.stdout:Restarting the monitor... 2024-03-26T17:51:46.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:45 smithi049 systemd[1]: Stopping Ceph mon.a for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:51:46.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a[31952]: 2024-03-26T17:51:46.140+0000 7f432b811700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T17:51:46.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a[31952]: 2024-03-26T17:51:46.140+0000 7f432b811700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-26T17:51:46.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:46 smithi049 podman[32203]: 2024-03-26 17:51:46.171988334 +0000 UTC m=+0.124253396 container died f70eef60af800fae8e77e7d91855b563e0c223646b4a2c1c0c9c47bda17f5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-26T17:51:46.665 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:46 smithi049 podman[32203]: 2024-03-26 17:51:46.490727544 +0000 UTC m=+0.442992603 container cleanup f70eef60af800fae8e77e7d91855b563e0c223646b4a2c1c0c9c47bda17f5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-26T17:51:46.665 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:46 smithi049 bash[32203]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a 2024-03-26T17:51:47.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:46 smithi049 podman[32217]: 2024-03-26 17:51:46.67508973 +0000 UTC m=+0.505350028 container remove f70eef60af800fae8e77e7d91855b563e0c223646b4a2c1c0c9c47bda17f5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T17:51:47.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:46 smithi049 systemd[1]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-26T17:51:47.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:46 smithi049 systemd[1]: Stopped Ceph mon.a for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:51:47.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:46 smithi049 systemd[1]: Starting Ceph mon.a for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:51:47.378 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 podman[32312]: 2024-03-26 17:51:47.117439317 +0000 UTC m=+0.112745396 container create f601912b7bc11471815d1a9309e870fb67109684e4edda065dcd5ed7da392b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-26T17:51:47.378 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 podman[32312]: 2024-03-26 17:51:47.023269806 +0000 UTC m=+0.018575886 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:51:47.378 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 podman[32312]: 2024-03-26 17:51:47.302874093 +0000 UTC m=+0.298180172 container init f601912b7bc11471815d1a9309e870fb67109684e4edda065dcd5ed7da392b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T17:51:47.378 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 podman[32312]: 2024-03-26 17:51:47.305772369 +0000 UTC m=+0.301078449 container start f601912b7bc11471815d1a9309e870fb67109684e4edda065dcd5ed7da392b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True) 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev), process ceph-mon, pid 2 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: pidfile_write: ignore empty --pid-file 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: load: jerasure load: lrc 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: RocksDB version: 7.9.2 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Git sha 0 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Compile date 2024-03-20 16:36:06 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: DB SUMMARY 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: DB Session ID: 0MYBRE3O0C0H5K2M265U 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: CURRENT file: CURRENT 2024-03-26T17:51:47.379 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105676 ; 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.error_if_exists: 0 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.create_if_missing: 0 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.paranoid_checks: 1 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.env: 0x559c53bb3c60 2024-03-26T17:51:47.380 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.info_log: 0x559c54a5bae0 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.statistics: (nil) 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.use_fsync: 0 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_log_file_size: 0 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.allow_fallocate: 1 2024-03-26T17:51:47.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T17:51:47.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T17:51:47.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.use_direct_reads: 0 2024-03-26T17:51:47.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T17:51:47.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T17:51:47.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.db_log_dir: 2024-03-26T17:51:47.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.wal_dir: 2024-03-26T17:51:47.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T17:51:47.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T17:51:47.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.write_buffer_manager: 0x559c54a2e1e0 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T17:51:47.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.unordered_write: 0 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.row_cache: None 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.wal_filter: None 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.two_write_queues: 0 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.wal_compression: 0 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.atomic_flush: 0 2024-03-26T17:51:47.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T17:51:47.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T17:51:47.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T17:51:47.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.log_readahead_size: 0 2024-03-26T17:51:47.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T17:51:47.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T17:51:47.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T17:51:47.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T17:51:47.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T17:51:47.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T17:51:47.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_background_jobs: 2 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_background_compactions: -1 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_subcompactions: 1 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T17:51:47.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_open_files: -1 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_background_flushes: -1 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Compression algorithms supported: 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: kZSTD supported: 0 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: kXpressCompression supported: 0 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: kZlibCompression supported: 1 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: kSnappyCompression supported: 1 2024-03-26T17:51:47.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: kLZ4Compression supported: 1 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: kBZip2Compression supported: 0 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.merge_operator: 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_filter: None 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559c54a5bbe0) 2024-03-26T17:51:47.388 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks: 1 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_top_level_index_and_filter: 1 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_type: 0 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_index_type: 0 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_shortening: 1 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: checksum: 4 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: no_block_cache: 0 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache: 0x559c54a61090 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_name: BinnedLRUCache 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_options: 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: capacity : 536870912 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_shard_bits : 4 2024-03-26T17:51:47.389 INFO:journalctl@ceph.mon.a.smithi049.stdout: strict_capacity_limit : 0 2024-03-26T17:51:47.390 INFO:journalctl@ceph.mon.a.smithi049.stdout: high_pri_pool_ratio: 0.000 2024-03-26T17:51:47.390 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_compressed: (nil) 2024-03-26T17:51:47.390 INFO:journalctl@ceph.mon.a.smithi049.stdout: persistent_cache: (nil) 2024-03-26T17:51:47.390 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size: 4096 2024-03-26T17:51:47.390 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size_deviation: 10 2024-03-26T17:51:47.390 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_restart_interval: 16 2024-03-26T17:51:47.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_block_restart_interval: 1 2024-03-26T17:51:47.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: metadata_block_size: 4096 2024-03-26T17:51:47.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: partition_filters: 0 2024-03-26T17:51:47.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: use_delta_encoding: 1 2024-03-26T17:51:47.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: filter_policy: bloomfilter 2024-03-26T17:51:47.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: whole_key_filtering: 1 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: verify_compression: 0 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: read_amp_bytes_per_bit: 0 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: format_version: 5 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_index_compression: 1 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_align: 0 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: max_auto_readahead_size: 262144 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: prepopulate_block_cache: 0 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: initial_auto_readahead_size: 8192 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compression: NoCompression 2024-03-26T17:51:47.392 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.num_levels: 7 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T17:51:47.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T17:51:47.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T17:51:47.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T17:51:47.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.table_properties_collectors: 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.inplace_update_support: 0 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T17:51:47.397 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.bloom_locality: 0 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.max_successive_merges: 0 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.ttl: 2592000 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.enable_blob_files: false 2024-03-26T17:51:47.398 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.min_blob_size: 0 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fcc32d56-9e25-43d7-a42c-b325012f0fe5 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711475507335573, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-26T17:51:47.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711475507336565, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100719, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98639, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14306, "raw_average_key_size": 49, "raw_value_size": 90885, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711475507, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fcc32d56-9e25-43d7-a42c-b325012f0fe5", "db_session_id": "0MYBRE3O0C0H5K2M265U", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711475507336650, "job": 1, "event": "recovery_finished"} 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559c54b54000 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: DB pointer 0x559c54b3e000 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** DB Stats ** 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T17:51:47.400 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: L0 2/0 100.19 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: Sum 2/0 100.19 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T17:51:47.401 INFO:journalctl@ceph.mon.a.smithi049.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 159.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative compaction: 0.00 GB write, 31.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T17:51:47.402 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval compaction: 0.00 GB write, 31.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache BinnedLRUCache@0x559c54a61090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: starting mon.a rank 0 at public addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] at bind addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: mon.a@-1(???) e1 preinit fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: mon.a@-1(???).mds e1 new map 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: mon.a@-1(???).mds e1 print_map 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout: e1 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T17:51:47.403 INFO:journalctl@ceph.mon.a.smithi049.stdout: legacy client fscid: -1 2024-03-26T17:51:47.404 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2024-03-26T17:51:47.404 INFO:journalctl@ceph.mon.a.smithi049.stdout: No filesystems configured 2024-03-26T17:51:47.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T17:51:47.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:51:47.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:51:47.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:51:47.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-26T17:51:47.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 bash[32312]: f601912b7bc11471815d1a9309e870fb67109684e4edda065dcd5ed7da392b52 2024-03-26T17:51:47.404 INFO:teuthology.orchestra.run.smithi049.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-26T17:51:47.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 systemd[1]: Started Ceph mon.a for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:51:47.669 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T17:51:47.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: monmap epoch 1 2024-03-26T17:51:47.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:51:47.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: last_changed 2024-03-26T17:51:39.354207+0000 2024-03-26T17:51:47.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: created 2024-03-26T17:51:39.354207+0000 2024-03-26T17:51:47.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: min_mon_release 19 (squid) 2024-03-26T17:51:47.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: election_strategy: 1 2024-03-26T17:51:47.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-26T17:51:47.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: fsmap 2024-03-26T17:51:47.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T17:51:47.671 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:47 smithi049 ceph-mon[32331]: mgrmap e1: no daemons active 2024-03-26T17:51:48.574 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.939+0000 7f7bd5605700 1 Processor -- start 2024-03-26T17:51:48.574 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.939+0000 7f7bd5605700 1 -- start start 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.939+0000 7f7bd5605700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd01083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.939+0000 7f7bd5605700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01088e0 con 0x7f7bd0107f80 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.939+0000 7f7bceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd01083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.939+0000 7f7bceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd01083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58830/0 (socket says 172.21.15.49:58830) 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.939+0000 7f7bceffd700 1 -- 172.21.15.49:0/2436009310 learned_addr learned my addr 172.21.15.49:0/2436009310 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.940+0000 7f7bceffd700 1 -- 172.21.15.49:0/2436009310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd0109090 con 0x7f7bd0107f80 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.940+0000 7f7bceffd700 1 --2- 172.21.15.49:0/2436009310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd01083a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00098a0 tx=0x7f7bc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb30edb2a5cae0f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.940+0000 7f7bcdffb700 1 -- 172.21.15.49:0/2436009310 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bc0004160 con 0x7f7bd0107f80 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.940+0000 7f7bcdffb700 1 -- 172.21.15.49:0/2436009310 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7bc0004300 con 0x7f7bd0107f80 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.940+0000 7f7bcdffb700 1 -- 172.21.15.49:0/2436009310 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bc0011b50 con 0x7f7bd0107f80 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.941+0000 7f7bd5605700 1 -- 172.21.15.49:0/2436009310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 msgr2=0x7f7bd01083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.941+0000 7f7bd5605700 1 --2- 172.21.15.49:0/2436009310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd01083a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00098a0 tx=0x7f7bc0009bb0 comp rx=0 tx=0).stop 2024-03-26T17:51:48.575 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.941+0000 7f7bd5605700 1 -- 172.21.15.49:0/2436009310 shutdown_connections 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.941+0000 7f7bd5605700 1 --2- 172.21.15.49:0/2436009310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd01083a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.941+0000 7f7bd5605700 1 -- 172.21.15.49:0/2436009310 >> 172.21.15.49:0/2436009310 conn(0x7f7bd0103540 msgr2=0x7f7bd0105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.941+0000 7f7bd5605700 1 -- 172.21.15.49:0/2436009310 shutdown_connections 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.941+0000 7f7bd5605700 1 -- 172.21.15.49:0/2436009310 wait complete. 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.941+0000 7f7bd5605700 1 Processor -- start 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.941+0000 7f7bd5605700 1 -- start start 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.942+0000 7f7bd5605700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd019be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.942+0000 7f7bd5605700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd010c4d0 con 0x7f7bd0107f80 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.942+0000 7f7bceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd019be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.942+0000 7f7bceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd019be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58838/0 (socket says 172.21.15.49:58838) 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.942+0000 7f7bceffd700 1 -- 172.21.15.49:0/2665010105 learned_addr learned my addr 172.21.15.49:0/2665010105 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.942+0000 7f7bceffd700 1 -- 172.21.15.49:0/2665010105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd019c3d0 con 0x7f7bd0107f80 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.942+0000 7f7bceffd700 1 --2- 172.21.15.49:0/2665010105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd019be90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0011670 tx=0x7f7bc0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:48.576 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.942+0000 7f7bb7fff700 1 -- 172.21.15.49:0/2665010105 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bc00039c0 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.942+0000 7f7bb7fff700 1 -- 172.21.15.49:0/2665010105 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7bc0011e50 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.942+0000 7f7bd5605700 1 -- 172.21.15.49:0/2665010105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd019c660 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.943+0000 7f7bb7fff700 1 -- 172.21.15.49:0/2665010105 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bc001a5c0 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.943+0000 7f7bd5605700 1 -- 172.21.15.49:0/2665010105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd019f350 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.943+0000 7f7bb7fff700 1 -- 172.21.15.49:0/2665010105 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7bc001ac00 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.943+0000 7f7bb7fff700 1 -- 172.21.15.49:0/2665010105 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7bc0019700 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.943+0000 7f7bd5605700 1 -- 172.21.15.49:0/2665010105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd00639e0 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:47.946+0000 7f7bb7fff700 1 -- 172.21.15.49:0/2665010105 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7bc0003b60 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:48.015+0000 7f7bd5605700 1 -- 172.21.15.49:0/2665010105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7bd019f520 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:48.017+0000 7f7bb7fff700 1 -- 172.21.15.49:0/2665010105 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7bc0019900 con 0x7f7bd0107f80 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:48.018+0000 7f7bd5605700 1 -- 172.21.15.49:0/2665010105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 msgr2=0x7f7bd019be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:48.577 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:48.018+0000 7f7bd5605700 1 --2- 172.21.15.49:0/2665010105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd019be90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0011670 tx=0x7f7bc0011b60 comp rx=0 tx=0).stop 2024-03-26T17:51:48.578 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:48.018+0000 7f7bd5605700 1 -- 172.21.15.49:0/2665010105 shutdown_connections 2024-03-26T17:51:48.578 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:48.018+0000 7f7bd5605700 1 --2- 172.21.15.49:0/2665010105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd0107f80 0x7f7bd019be90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:48.578 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:48.018+0000 7f7bd5605700 1 -- 172.21.15.49:0/2665010105 >> 172.21.15.49:0/2665010105 conn(0x7f7bd0103540 msgr2=0x7f7bd0103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:48.578 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:48.018+0000 7f7bd5605700 1 -- 172.21.15.49:0/2665010105 shutdown_connections 2024-03-26T17:51:48.578 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:48.018+0000 7f7bd5605700 1 -- 172.21.15.49:0/2665010105 wait complete. 2024-03-26T17:51:48.578 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-26T17:51:48.608 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-26T17:51:48.608 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mgr... 2024-03-26T17:51:48.609 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-26T17:51:48.609 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-26T17:51:48.869 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.x 2024-03-26T17:51:48.870 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-26T17:51:49.023 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5a3632a4-eb99-11ee-95d0-87774f69a715.target.wants/ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@.service. 2024-03-26T17:51:49.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:49 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2665010105' entity='client.admin' 2024-03-26T17:51:49.252 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:49 smithi049 systemd[1]: Starting Ceph mgr.x for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:51:49.511 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:49 smithi049 podman[32547]: 2024-03-26 17:51:49.261776841 +0000 UTC m=+0.109558520 container create 819b241b3d514701469d7787a79d6180f4ea596c1d36e065aac1aeb5f1762f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-26T17:51:49.511 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:49 smithi049 podman[32547]: 2024-03-26 17:51:49.170469075 +0000 UTC m=+0.018250770 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:51:49.511 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:49 smithi049 podman[32547]: 2024-03-26 17:51:49.489283056 +0000 UTC m=+0.337064737 container init 819b241b3d514701469d7787a79d6180f4ea596c1d36e065aac1aeb5f1762f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T17:51:49.511 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:49 smithi049 podman[32547]: 2024-03-26 17:51:49.491818957 +0000 UTC m=+0.339600641 container start 819b241b3d514701469d7787a79d6180f4ea596c1d36e065aac1aeb5f1762f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-26T17:51:49.570 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T17:51:49.570 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-26T17:51:49.578 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T17:51:49.578 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-26T17:51:49.578 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-26T17:51:49.578 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T17:51:49.586 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T17:51:49.586 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-26T17:51:49.594 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T17:51:49.594 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-26T17:51:49.594 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-26T17:51:49.594 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-26T17:51:49.594 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr to start... 2024-03-26T17:51:49.594 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr... 2024-03-26T17:51:49.814 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:49 smithi049 bash[32547]: 819b241b3d514701469d7787a79d6180f4ea596c1d36e065aac1aeb5f1762f89 2024-03-26T17:51:49.814 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:49 smithi049 systemd[1]: Started Ceph mgr.x for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:51:50.083 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:50 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:50.069+0000 7fca99344200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T17:51:50.427 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:50 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1547939632' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T17:51:50.427 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:50 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:50.169+0000 7fca99344200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T17:51:50.908 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T17:51:50.909 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T17:51:50.910 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T17:51:50.911 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T17:51:42.368470+0000", 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.010+0000 7fb52575f700 1 Processor -- start 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.010+0000 7fb52575f700 1 -- start start 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.010+0000 7fb52575f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb520108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:50.912 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.010+0000 7fb52575f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5201088d0 con 0x7fb520107f70 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.011+0000 7fb51effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb520108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.011+0000 7fb51effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb520108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44176/0 (socket says 172.21.15.49:44176) 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.011+0000 7fb51effd700 1 -- 172.21.15.49:0/161420256 learned_addr learned my addr 172.21.15.49:0/161420256 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.011+0000 7fb51effd700 1 -- 172.21.15.49:0/161420256 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb520109080 con 0x7fb520107f70 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.011+0000 7fb51effd700 1 --2- 172.21.15.49:0/161420256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb520108390 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb5140098a0 tx=0x7fb514009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bad6648db78a2cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.011+0000 7fb51dffb700 1 -- 172.21.15.49:0/161420256 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb514004160 con 0x7fb520107f70 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.012+0000 7fb51dffb700 1 -- 172.21.15.49:0/161420256 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb514004300 con 0x7fb520107f70 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.012+0000 7fb51dffb700 1 -- 172.21.15.49:0/161420256 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb514011b50 con 0x7fb520107f70 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.012+0000 7fb52575f700 1 -- 172.21.15.49:0/161420256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 msgr2=0x7fb520108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.012+0000 7fb52575f700 1 --2- 172.21.15.49:0/161420256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb520108390 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb5140098a0 tx=0x7fb514009bb0 comp rx=0 tx=0).stop 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.012+0000 7fb52575f700 1 -- 172.21.15.49:0/161420256 shutdown_connections 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.012+0000 7fb52575f700 1 --2- 172.21.15.49:0/161420256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb520108390 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.012+0000 7fb52575f700 1 -- 172.21.15.49:0/161420256 >> 172.21.15.49:0/161420256 conn(0x7fb520103510 msgr2=0x7fb520105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:50.913 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.013+0000 7fb52575f700 1 -- 172.21.15.49:0/161420256 shutdown_connections 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.013+0000 7fb52575f700 1 -- 172.21.15.49:0/161420256 wait complete. 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.013+0000 7fb52575f700 1 Processor -- start 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.014+0000 7fb52575f700 1 -- start start 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.014+0000 7fb52575f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb52019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.014+0000 7fb52575f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb52010c4c0 con 0x7fb520107f70 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.014+0000 7fb51effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb52019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.014+0000 7fb51effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb52019be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44178/0 (socket says 172.21.15.49:44178) 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.014+0000 7fb51effd700 1 -- 172.21.15.49:0/1547939632 learned_addr learned my addr 172.21.15.49:0/1547939632 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.014+0000 7fb51effd700 1 -- 172.21.15.49:0/1547939632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52019c380 con 0x7fb520107f70 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.014+0000 7fb51effd700 1 --2- 172.21.15.49:0/1547939632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb52019be40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb514011b60 tx=0x7fb514011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.015+0000 7fb507fff700 1 -- 172.21.15.49:0/1547939632 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb514003b20 con 0x7fb520107f70 2024-03-26T17:51:50.914 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.015+0000 7fb52575f700 1 -- 172.21.15.49:0/1547939632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52019c610 con 0x7fb520107f70 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.015+0000 7fb507fff700 1 -- 172.21.15.49:0/1547939632 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb514003cc0 con 0x7fb520107f70 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.015+0000 7fb507fff700 1 -- 172.21.15.49:0/1547939632 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb514024840 con 0x7fb520107f70 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.015+0000 7fb52575f700 1 -- 172.21.15.49:0/1547939632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52019f300 con 0x7fb520107f70 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.015+0000 7fb507fff700 1 -- 172.21.15.49:0/1547939632 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb514022020 con 0x7fb520107f70 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.015+0000 7fb507fff700 1 -- 172.21.15.49:0/1547939632 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb514011e00 con 0x7fb520107f70 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.016+0000 7fb52575f700 1 -- 172.21.15.49:0/1547939632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5200694b0 con 0x7fb520107f70 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.018+0000 7fb507fff700 1 -- 172.21.15.49:0/1547939632 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb5140193c0 con 0x7fb520107f70 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.092+0000 7fb52575f700 1 -- 172.21.15.49:0/1547939632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb52019f8c0 con 0x7fb520107f70 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.092+0000 7fb507fff700 1 -- 172.21.15.49:0/1547939632 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb5140195a0 con 0x7fb520107f70 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.094+0000 7fb52575f700 1 -- 172.21.15.49:0/1547939632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 msgr2=0x7fb52019be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.094+0000 7fb52575f700 1 --2- 172.21.15.49:0/1547939632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb52019be40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb514011b60 tx=0x7fb514011c40 comp rx=0 tx=0).stop 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.094+0000 7fb52575f700 1 -- 172.21.15.49:0/1547939632 shutdown_connections 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.094+0000 7fb52575f700 1 --2- 172.21.15.49:0/1547939632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb520107f70 0x7fb52019be40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:50.915 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.094+0000 7fb52575f700 1 -- 172.21.15.49:0/1547939632 >> 172.21.15.49:0/1547939632 conn(0x7fb520103510 msgr2=0x7fb520103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:50.916 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.094+0000 7fb52575f700 1 -- 172.21.15.49:0/1547939632 shutdown_connections 2024-03-26T17:51:50.916 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:50.094+0000 7fb52575f700 1 -- 172.21.15.49:0/1547939632 wait complete. 2024-03-26T17:51:50.916 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (1/15)... 2024-03-26T17:51:51.232 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:50 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:50.895+0000 7fca99344200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T17:51:51.232 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:50 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:50.996+0000 7fca99344200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T17:51:51.232 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:51.142+0000 7fca99344200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T17:51:51.232 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:51.240+0000 7fca99344200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T17:51:51.511 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:51.327+0000 7fca99344200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T17:51:51.892 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:51.602+0000 7fca99344200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T17:51:51.892 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:51.716+0000 7fca99344200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T17:51:51.892 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:51.803+0000 7fca99344200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T17:51:51.892 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:51.901+0000 7fca99344200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T17:51:52.261 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:52 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:52.085+0000 7fca99344200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T17:51:53.292 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:53 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:53.127+0000 7fca99344200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T17:51:53.293 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:53 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:53.267+0000 7fca99344200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T17:51:53.614 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:53 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1093150700' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T17:51:53.912 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:53 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:53.622+0000 7fca99344200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T17:51:53.912 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:53 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:53.717+0000 7fca99344200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T17:51:53.912 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:53 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:53.921+0000 7fca99344200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:51:53.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:51:53.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:51:53.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T17:51:53.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T17:51:53.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T17:51:53.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T17:51:53.976 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:53.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T17:51:53.978 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T17:51:53.979 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T17:51:42.368470+0000", 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T17:51:53.980 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.328+0000 7f1132c16700 1 Processor -- start 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.328+0000 7f1132c16700 1 -- start start 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.328+0000 7f1132c16700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c10ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.328+0000 7f1132c16700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f112c07bda0 con 0x7f112c1089f0 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.328+0000 7f11309b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c10ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.328+0000 7f11309b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c10ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44186/0 (socket says 172.21.15.49:44186) 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.328+0000 7f11309b2700 1 -- 172.21.15.49:0/888063418 learned_addr learned my addr 172.21.15.49:0/888063418 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.329+0000 7f11309b2700 1 -- 172.21.15.49:0/888063418 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c10b320 con 0x7f112c1089f0 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.329+0000 7f11309b2700 1 --2- 172.21.15.49:0/888063418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c10ade0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11200098a0 tx=0x7f1120009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d08cb8684cd4d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.329+0000 7f112b7fe700 1 -- 172.21.15.49:0/888063418 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1120004160 con 0x7f112c1089f0 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.329+0000 7f112b7fe700 1 -- 172.21.15.49:0/888063418 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1120004300 con 0x7f112c1089f0 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.329+0000 7f112b7fe700 1 -- 172.21.15.49:0/888063418 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1120011b50 con 0x7f112c1089f0 2024-03-26T17:51:53.981 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.330+0000 7f1132c16700 1 -- 172.21.15.49:0/888063418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 msgr2=0x7f112c10ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.330+0000 7f1132c16700 1 --2- 172.21.15.49:0/888063418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c10ade0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11200098a0 tx=0x7f1120009bb0 comp rx=0 tx=0).stop 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.330+0000 7f1132c16700 1 -- 172.21.15.49:0/888063418 shutdown_connections 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.330+0000 7f1132c16700 1 --2- 172.21.15.49:0/888063418 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c10ade0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.330+0000 7f1132c16700 1 -- 172.21.15.49:0/888063418 >> 172.21.15.49:0/888063418 conn(0x7f112c102420 msgr2=0x7f112c104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.330+0000 7f1132c16700 1 -- 172.21.15.49:0/888063418 shutdown_connections 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.330+0000 7f1132c16700 1 -- 172.21.15.49:0/888063418 wait complete. 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.330+0000 7f1132c16700 1 Processor -- start 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.330+0000 7f1132c16700 1 -- start start 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f1132c16700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f1132c16700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f112c10c340 con 0x7f112c1089f0 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f11309b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f11309b2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c19be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44198/0 (socket says 172.21.15.49:44198) 2024-03-26T17:51:53.982 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f11309b2700 1 -- 172.21.15.49:0/1093150700 learned_addr learned my addr 172.21.15.49:0/1093150700 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:53.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f11309b2700 1 -- 172.21.15.49:0/1093150700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c19c3c0 con 0x7f112c1089f0 2024-03-26T17:51:53.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f11309b2700 1 --2- 172.21.15.49:0/1093150700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c19be80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1120011670 tx=0x7f1120011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:53.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f1129ffb700 1 -- 172.21.15.49:0/1093150700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11200039c0 con 0x7f112c1089f0 2024-03-26T17:51:53.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f1129ffb700 1 -- 172.21.15.49:0/1093150700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1120011e50 con 0x7f112c1089f0 2024-03-26T17:51:53.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f1132c16700 1 -- 172.21.15.49:0/1093150700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112c19c650 con 0x7f112c1089f0 2024-03-26T17:51:53.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.331+0000 7f1129ffb700 1 -- 172.21.15.49:0/1093150700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f112001a5c0 con 0x7f112c1089f0 2024-03-26T17:51:53.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.332+0000 7f1132c16700 1 -- 172.21.15.49:0/1093150700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f112c19f340 con 0x7f112c1089f0 2024-03-26T17:51:53.983 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.332+0000 7f1129ffb700 1 -- 172.21.15.49:0/1093150700 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f112001ac00 con 0x7f112c1089f0 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.332+0000 7f1129ffb700 1 -- 172.21.15.49:0/1093150700 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1120019700 con 0x7f112c1089f0 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.333+0000 7f1132c16700 1 -- 172.21.15.49:0/1093150700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f112c0694b0 con 0x7f112c1089f0 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.335+0000 7f1129ffb700 1 -- 172.21.15.49:0/1093150700 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1120003b60 con 0x7f112c1089f0 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.409+0000 7f1132c16700 1 -- 172.21.15.49:0/1093150700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f112c19f510 con 0x7f112c1089f0 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.410+0000 7f1129ffb700 1 -- 172.21.15.49:0/1093150700 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1120019900 con 0x7f112c1089f0 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.411+0000 7f1132c16700 1 -- 172.21.15.49:0/1093150700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 msgr2=0x7f112c19be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.411+0000 7f1132c16700 1 --2- 172.21.15.49:0/1093150700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c19be80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1120011670 tx=0x7f1120011b60 comp rx=0 tx=0).stop 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.411+0000 7f1132c16700 1 -- 172.21.15.49:0/1093150700 shutdown_connections 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.411+0000 7f1132c16700 1 --2- 172.21.15.49:0/1093150700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f112c1089f0 0x7f112c19be80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.411+0000 7f1132c16700 1 -- 172.21.15.49:0/1093150700 >> 172.21.15.49:0/1093150700 conn(0x7f112c102420 msgr2=0x7f112c102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.411+0000 7f1132c16700 1 -- 172.21.15.49:0/1093150700 shutdown_connections 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:53.411+0000 7f1132c16700 1 -- 172.21.15.49:0/1093150700 wait complete. 2024-03-26T17:51:53.984 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (2/15)... 2024-03-26T17:51:54.261 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:54 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:54.113+0000 7fca99344200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T17:51:54.695 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:54 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:54.411+0000 7fca99344200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:51:54.696 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:54 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:54.704+0000 7fca99344200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T17:51:55.011 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:54 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:54.950+0000 7fca99344200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T17:51:56.100 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:55 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:55.794+0000 7fca99344200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T17:51:56.720 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:56 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3654217329' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T17:51:56.720 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:56 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:56.394+0000 7fca99344200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T17:51:56.721 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:56 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:56.500+0000 7fca99344200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T17:51:56.721 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:51:56 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:51:56.729+0000 7fca99344200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T17:51:58.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: Activating manager daemon x 2024-03-26T17:51:58.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: mgrmap e2: x(active, starting, since 0.00286821s) 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: from='mgr.14100 172.21.15.49:0/1313177497' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: from='mgr.14100 172.21.15.49:0/1313177497' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: from='mgr.14100 172.21.15.49:0/1313177497' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: from='mgr.14100 172.21.15.49:0/1313177497' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: from='mgr.14100 172.21.15.49:0/1313177497' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: Manager daemon x is now available 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: from='mgr.14100 172.21.15.49:0/1313177497' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: from='mgr.14100 172.21.15.49:0/1313177497' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: from='mgr.14100 172.21.15.49:0/1313177497' entity='mgr.x' 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: from='mgr.14100 172.21.15.49:0/1313177497' entity='mgr.x' 2024-03-26T17:51:58.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:57 smithi049 ceph-mon[32331]: from='mgr.14100 172.21.15.49:0/1313177497' entity='mgr.x' 2024-03-26T17:51:59.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:51:58 smithi049 ceph-mon[32331]: mgrmap e3: x(active, since 1.00715s) 2024-03-26T17:51:59.654 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-26T17:51:59.655 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T17:51:59.656 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T17:51:59.657 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T17:51:59.658 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T17:51:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T17:51:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:51:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:51:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T17:51:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:51:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T17:51:42.368470+0000", 2024-03-26T17:51:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:51:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:51:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.401+0000 7ff9d569e700 1 Processor -- start 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.401+0000 7ff9d569e700 1 -- start start 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.401+0000 7ff9d569e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d010add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.401+0000 7ff9d569e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d007c630 con 0x7ff9d01089e0 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.401+0000 7ff9ceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d010add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.401+0000 7ff9ceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d010add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44204/0 (socket says 172.21.15.49:44204) 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.401+0000 7ff9ceffd700 1 -- 172.21.15.49:0/2450174416 learned_addr learned my addr 172.21.15.49:0/2450174416 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.401+0000 7ff9ceffd700 1 -- 172.21.15.49:0/2450174416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d010b310 con 0x7ff9d01089e0 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.401+0000 7ff9ceffd700 1 --2- 172.21.15.49:0/2450174416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d010add0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00098a0 tx=0x7ff9c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1721a056e4fbd6b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.402+0000 7ff9cdffb700 1 -- 172.21.15.49:0/2450174416 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9c0004160 con 0x7ff9d01089e0 2024-03-26T17:51:59.660 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.402+0000 7ff9cdffb700 1 -- 172.21.15.49:0/2450174416 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9c0004300 con 0x7ff9d01089e0 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.402+0000 7ff9cdffb700 1 -- 172.21.15.49:0/2450174416 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9c0011b50 con 0x7ff9d01089e0 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.402+0000 7ff9d569e700 1 -- 172.21.15.49:0/2450174416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 msgr2=0x7ff9d010add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.402+0000 7ff9d569e700 1 --2- 172.21.15.49:0/2450174416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d010add0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00098a0 tx=0x7ff9c0009bb0 comp rx=0 tx=0).stop 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.402+0000 7ff9d569e700 1 -- 172.21.15.49:0/2450174416 shutdown_connections 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.402+0000 7ff9d569e700 1 --2- 172.21.15.49:0/2450174416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d010add0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.402+0000 7ff9d569e700 1 -- 172.21.15.49:0/2450174416 >> 172.21.15.49:0/2450174416 conn(0x7ff9d0102450 msgr2=0x7ff9d0104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.402+0000 7ff9d569e700 1 -- 172.21.15.49:0/2450174416 shutdown_connections 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.403+0000 7ff9d569e700 1 -- 172.21.15.49:0/2450174416 wait complete. 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.403+0000 7ff9d569e700 1 Processor -- start 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.403+0000 7ff9d569e700 1 -- start start 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.403+0000 7ff9d569e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.403+0000 7ff9d569e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d010c330 con 0x7ff9d01089e0 2024-03-26T17:51:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9ceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9ceffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d019bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44216/0 (socket says 172.21.15.49:44216) 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9ceffd700 1 -- 172.21.15.49:0/3654217329 learned_addr learned my addr 172.21.15.49:0/3654217329 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9ceffd700 1 -- 172.21.15.49:0/3654217329 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d019c420 con 0x7ff9d01089e0 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9ceffd700 1 --2- 172.21.15.49:0/3654217329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d019bee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009fb0 tx=0x7ff9c0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9b7fff700 1 -- 172.21.15.49:0/3654217329 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9c00038b0 con 0x7ff9d01089e0 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9d569e700 1 -- 172.21.15.49:0/3654217329 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d019c6b0 con 0x7ff9d01089e0 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9b7fff700 1 -- 172.21.15.49:0/3654217329 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9c0011e50 con 0x7ff9d01089e0 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9b7fff700 1 -- 172.21.15.49:0/3654217329 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9c0019420 con 0x7ff9d01089e0 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9d569e700 1 -- 172.21.15.49:0/3654217329 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d019f3a0 con 0x7ff9d01089e0 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.404+0000 7ff9b7fff700 1 -- 172.21.15.49:0/3654217329 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff9c0019ad0 con 0x7ff9d01089e0 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.405+0000 7ff9b7fff700 1 -- 172.21.15.49:0/3654217329 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff9c0024730 con 0x7ff9d01089e0 2024-03-26T17:51:59.662 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.405+0000 7ff9d569e700 1 -- 172.21.15.49:0/3654217329 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d00694b0 con 0x7ff9d01089e0 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.407+0000 7ff9b7fff700 1 -- 172.21.15.49:0/3654217329 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff9c0043dd0 con 0x7ff9d01089e0 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.481+0000 7ff9d569e700 1 -- 172.21.15.49:0/3654217329 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff9d019f840 con 0x7ff9d01089e0 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.481+0000 7ff9b7fff700 1 -- 172.21.15.49:0/3654217329 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff9c0032030 con 0x7ff9d01089e0 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.482+0000 7ff9d569e700 1 -- 172.21.15.49:0/3654217329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 msgr2=0x7ff9d019bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.482+0000 7ff9d569e700 1 --2- 172.21.15.49:0/3654217329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d019bee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009fb0 tx=0x7ff9c0011bf0 comp rx=0 tx=0).stop 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.482+0000 7ff9d569e700 1 -- 172.21.15.49:0/3654217329 shutdown_connections 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.482+0000 7ff9d569e700 1 --2- 172.21.15.49:0/3654217329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9d01089e0 0x7ff9d019bee0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.482+0000 7ff9d569e700 1 -- 172.21.15.49:0/3654217329 >> 172.21.15.49:0/3654217329 conn(0x7ff9d0102450 msgr2=0x7ff9d0102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.483+0000 7ff9d569e700 1 -- 172.21.15.49:0/3654217329 shutdown_connections 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:51:56.483+0000 7ff9d569e700 1 -- 172.21.15.49:0/3654217329 wait complete. 2024-03-26T17:51:59.663 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (3/15)... 2024-03-26T17:52:01.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:00 smithi049 ceph-mon[32331]: mgrmap e4: x(active, since 3s) 2024-03-26T17:52:06.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:05 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3891294778' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T17:52:06.416 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T17:52:06.417 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T17:52:06.418 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T17:52:06.427 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T17:52:06.427 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T17:52:06.427 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T17:52:06.427 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:52:06.427 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T17:52:06.427 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T17:52:06.427 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T17:52:06.427 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T17:52:06.427 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T17:52:06.428 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T17:51:42.368470+0000", 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.554+0000 7f4db36f3700 1 Processor -- start 2024-03-26T17:52:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.554+0000 7f4db36f3700 1 -- start start 2024-03-26T17:52:06.430 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.555+0000 7f4db36f3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:06.430 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.555+0000 7f4db36f3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac1088d0 con 0x7f4dac107f70 2024-03-26T17:52:06.430 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.555+0000 7f4db148f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:06.430 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.555+0000 7f4db148f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48374/0 (socket says 172.21.15.49:48374) 2024-03-26T17:52:06.430 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.555+0000 7f4db148f700 1 -- 172.21.15.49:0/140938935 learned_addr learned my addr 172.21.15.49:0/140938935 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:06.430 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.555+0000 7f4db148f700 1 -- 172.21.15.49:0/140938935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac109080 con 0x7f4dac107f70 2024-03-26T17:52:06.431 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.555+0000 7f4db148f700 1 --2- 172.21.15.49:0/140938935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac108390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009e80 tx=0x7f4da400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106be9c177cfe8ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:06.431 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.556+0000 7f4dabfff700 1 -- 172.21.15.49:0/140938935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da4017070 con 0x7f4dac107f70 2024-03-26T17:52:06.431 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.556+0000 7f4dabfff700 1 -- 172.21.15.49:0/140938935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4da400b8c0 con 0x7f4dac107f70 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.556+0000 7f4dabfff700 1 -- 172.21.15.49:0/140938935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da40045c0 con 0x7f4dac107f70 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.556+0000 7f4db36f3700 1 -- 172.21.15.49:0/140938935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 msgr2=0x7f4dac108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.556+0000 7f4db36f3700 1 --2- 172.21.15.49:0/140938935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac108390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009e80 tx=0x7f4da400b330 comp rx=0 tx=0).stop 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.556+0000 7f4db36f3700 1 -- 172.21.15.49:0/140938935 shutdown_connections 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.556+0000 7f4db36f3700 1 --2- 172.21.15.49:0/140938935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac108390 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.556+0000 7f4db36f3700 1 -- 172.21.15.49:0/140938935 >> 172.21.15.49:0/140938935 conn(0x7f4dac103510 msgr2=0x7f4dac105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.556+0000 7f4db36f3700 1 -- 172.21.15.49:0/140938935 shutdown_connections 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.557+0000 7f4db36f3700 1 -- 172.21.15.49:0/140938935 wait complete. 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.557+0000 7f4db36f3700 1 Processor -- start 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.557+0000 7f4db36f3700 1 -- start start 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.557+0000 7f4db36f3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac07e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:06.432 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.557+0000 7f4db36f3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac10c4c0 con 0x7f4dac107f70 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.557+0000 7f4db148f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac07e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.557+0000 7f4db148f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac07e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48376/0 (socket says 172.21.15.49:48376) 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.557+0000 7f4db148f700 1 -- 172.21.15.49:0/3891294778 learned_addr learned my addr 172.21.15.49:0/3891294778 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.558+0000 7f4db148f700 1 -- 172.21.15.49:0/3891294778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac07e870 con 0x7f4dac107f70 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.558+0000 7f4db148f700 1 --2- 172.21.15.49:0/3891294778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac07e330 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4da40072a0 tx=0x7f4da400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.558+0000 7f4daa7fc700 1 -- 172.21.15.49:0/3891294778 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da4020020 con 0x7f4dac107f70 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.558+0000 7f4daa7fc700 1 -- 172.21.15.49:0/3891294778 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4da4003b50 con 0x7f4dac107f70 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.558+0000 7f4db36f3700 1 -- 172.21.15.49:0/3891294778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dac07ca60 con 0x7f4dac107f70 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.558+0000 7f4daa7fc700 1 -- 172.21.15.49:0/3891294778 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da4017040 con 0x7f4dac107f70 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.558+0000 7f4db36f3700 1 -- 172.21.15.49:0/3891294778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dac07cfa0 con 0x7f4dac107f70 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.559+0000 7f4daa7fc700 1 -- 172.21.15.49:0/3891294778 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 46283+0+0 (secure 0 0 0) 0x7f4da4023020 con 0x7f4dac107f70 2024-03-26T17:52:06.433 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.559+0000 7f4db36f3700 1 -- 172.21.15.49:0/3891294778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dac0694b0 con 0x7f4dac107f70 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.559+0000 7f4daa7fc700 1 --2- 172.21.15.49:0/3891294778 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f4d94039130 0x7f4d9403b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.559+0000 7f4daa7fc700 1 -- 172.21.15.49:0/3891294778 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4da400f050 con 0x7f4dac107f70 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.559+0000 7f4db0c8e700 1 --2- 172.21.15.49:0/3891294778 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f4d94039130 0x7f4d9403b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.560+0000 7f4db0c8e700 1 --2- 172.21.15.49:0/3891294778 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f4d94039130 0x7f4d9403b5f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4da0006fd0 tx=0x7f4da0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.565+0000 7f4daa7fc700 1 -- 172.21.15.49:0/3891294778 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4da402a020 con 0x7f4dac107f70 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.853+0000 7f4db36f3700 1 -- 172.21.15.49:0/3891294778 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4dac07d8e0 con 0x7f4dac107f70 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.854+0000 7f4daa7fc700 1 -- 172.21.15.49:0/3891294778 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4da402d350 con 0x7f4dac107f70 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.856+0000 7f4db36f3700 1 -- 172.21.15.49:0/3891294778 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f4d94039130 msgr2=0x7f4d9403b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.856+0000 7f4db36f3700 1 --2- 172.21.15.49:0/3891294778 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f4d94039130 0x7f4d9403b5f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4da0006fd0 tx=0x7f4da0006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.856+0000 7f4db36f3700 1 -- 172.21.15.49:0/3891294778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 msgr2=0x7f4dac07e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.856+0000 7f4db36f3700 1 --2- 172.21.15.49:0/3891294778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac07e330 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4da40072a0 tx=0x7f4da400bd50 comp rx=0 tx=0).stop 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.856+0000 7f4db36f3700 1 -- 172.21.15.49:0/3891294778 shutdown_connections 2024-03-26T17:52:06.434 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.856+0000 7f4db36f3700 1 --2- 172.21.15.49:0/3891294778 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f4d94039130 0x7f4d9403b5f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:06.435 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.856+0000 7f4db36f3700 1 --2- 172.21.15.49:0/3891294778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4dac107f70 0x7f4dac07e330 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:06.435 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.856+0000 7f4db36f3700 1 -- 172.21.15.49:0/3891294778 >> 172.21.15.49:0/3891294778 conn(0x7f4dac103510 msgr2=0x7f4dac103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:06.435 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.856+0000 7f4db36f3700 1 -- 172.21.15.49:0/3891294778 shutdown_connections 2024-03-26T17:52:06.435 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:05.856+0000 7f4db36f3700 1 -- 172.21.15.49:0/3891294778 wait complete. 2024-03-26T17:52:06.435 INFO:teuthology.orchestra.run.smithi049.stdout:mgr is available 2024-03-26T17:52:07.378 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:07 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3877939832' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T17:52:07.586 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:52:07.586 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T17:52:07.586 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:52:07.586 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T17:52:07.586 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T17:52:07.586 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T17:52:07.586 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T17:52:07.586 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:52:07.586 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.885+0000 7f44d3c51700 1 Processor -- start 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.885+0000 7f44d3c51700 1 -- start start 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.885+0000 7f44d3c51700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc108310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.885+0000 7f44d3c51700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc108850 con 0x7f44cc107ef0 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.886+0000 7f44d19ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:07.587 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.886+0000 7f44d19ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc108310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48390/0 (socket says 172.21.15.49:48390) 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.886+0000 7f44d19ed700 1 -- 172.21.15.49:0/341568968 learned_addr learned my addr 172.21.15.49:0/341568968 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.886+0000 7f44d19ed700 1 -- 172.21.15.49:0/341568968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44cc109090 con 0x7f44cc107ef0 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.886+0000 7f44d19ed700 1 --2- 172.21.15.49:0/341568968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc108310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f44c80098a0 tx=0x7f44c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf09429395c21dfe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.886+0000 7f44d09eb700 1 -- 172.21.15.49:0/341568968 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44c8004160 con 0x7f44cc107ef0 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.886+0000 7f44d09eb700 1 -- 172.21.15.49:0/341568968 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f44c8004300 con 0x7f44cc107ef0 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.886+0000 7f44d09eb700 1 -- 172.21.15.49:0/341568968 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44c8011b50 con 0x7f44cc107ef0 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.887+0000 7f44d3c51700 1 -- 172.21.15.49:0/341568968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 msgr2=0x7f44cc108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.887+0000 7f44d3c51700 1 --2- 172.21.15.49:0/341568968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc108310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f44c80098a0 tx=0x7f44c8009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.887+0000 7f44d3c51700 1 -- 172.21.15.49:0/341568968 shutdown_connections 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.887+0000 7f44d3c51700 1 --2- 172.21.15.49:0/341568968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc108310 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.887+0000 7f44d3c51700 1 -- 172.21.15.49:0/341568968 >> 172.21.15.49:0/341568968 conn(0x7f44cc103500 msgr2=0x7f44cc105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.887+0000 7f44d3c51700 1 -- 172.21.15.49:0/341568968 shutdown_connections 2024-03-26T17:52:07.588 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.887+0000 7f44d3c51700 1 -- 172.21.15.49:0/341568968 wait complete. 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.888+0000 7f44d3c51700 1 Processor -- start 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.888+0000 7f44d3c51700 1 -- start start 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.888+0000 7f44d3c51700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc07e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.888+0000 7f44d3c51700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10c350 con 0x7f44cc107ef0 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.888+0000 7f44d19ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc07e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.888+0000 7f44d19ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc07e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48392/0 (socket says 172.21.15.49:48392) 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.888+0000 7f44d19ed700 1 -- 172.21.15.49:0/3877939832 learned_addr learned my addr 172.21.15.49:0/3877939832 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.888+0000 7f44d19ed700 1 -- 172.21.15.49:0/3877939832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44cc07e870 con 0x7f44cc107ef0 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.888+0000 7f44d19ed700 1 --2- 172.21.15.49:0/3877939832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc07e330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009fb0 tx=0x7f44c8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.889+0000 7f44c2ffd700 1 -- 172.21.15.49:0/3877939832 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44c80038b0 con 0x7f44cc107ef0 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.889+0000 7f44c2ffd700 1 -- 172.21.15.49:0/3877939832 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f44c8011e50 con 0x7f44cc107ef0 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.889+0000 7f44d3c51700 1 -- 172.21.15.49:0/3877939832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44cc07ca60 con 0x7f44cc107ef0 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.889+0000 7f44c2ffd700 1 -- 172.21.15.49:0/3877939832 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44c8019550 con 0x7f44cc107ef0 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.889+0000 7f44d3c51700 1 -- 172.21.15.49:0/3877939832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44cc07cfa0 con 0x7f44cc107ef0 2024-03-26T17:52:07.589 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.889+0000 7f44c2ffd700 1 -- 172.21.15.49:0/3877939832 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 46283+0+0 (secure 0 0 0) 0x7f44c8019850 con 0x7f44cc107ef0 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.890+0000 7f44c2ffd700 1 --2- 172.21.15.49:0/3877939832 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f44b80390e0 0x7f44b803b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.890+0000 7f44c2ffd700 1 -- 172.21.15.49:0/3877939832 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f44c804d3d0 con 0x7f44cc107ef0 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.890+0000 7f44d11ec700 1 --2- 172.21.15.49:0/3877939832 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f44b80390e0 0x7f44b803b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.890+0000 7f44d3c51700 1 -- 172.21.15.49:0/3877939832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44cc0694b0 con 0x7f44cc107ef0 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.890+0000 7f44d11ec700 1 --2- 172.21.15.49:0/3877939832 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f44b80390e0 0x7f44b803b5a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006fd0 tx=0x7f44bc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:06.895+0000 7f44c2ffd700 1 -- 172.21.15.49:0/3877939832 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f44c804cdf0 con 0x7f44cc107ef0 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.097+0000 7f44d3c51700 1 -- 172.21.15.49:0/3877939832 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f44cc07d8d0 con 0x7f44cc107ef0 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.098+0000 7f44c2ffd700 1 -- 172.21.15.49:0/3877939832 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f44c8016dc0 con 0x7f44cc107ef0 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.100+0000 7f44d3c51700 1 -- 172.21.15.49:0/3877939832 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f44b80390e0 msgr2=0x7f44b803b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.100+0000 7f44d3c51700 1 --2- 172.21.15.49:0/3877939832 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f44b80390e0 0x7f44b803b5a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006fd0 tx=0x7f44bc006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.100+0000 7f44d3c51700 1 -- 172.21.15.49:0/3877939832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 msgr2=0x7f44cc07e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.100+0000 7f44d3c51700 1 --2- 172.21.15.49:0/3877939832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc07e330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009fb0 tx=0x7f44c8011bf0 comp rx=0 tx=0).stop 2024-03-26T17:52:07.590 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.100+0000 7f44d3c51700 1 -- 172.21.15.49:0/3877939832 shutdown_connections 2024-03-26T17:52:07.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.100+0000 7f44d3c51700 1 --2- 172.21.15.49:0/3877939832 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f44b80390e0 0x7f44b803b5a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:07.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.100+0000 7f44d3c51700 1 --2- 172.21.15.49:0/3877939832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44cc107ef0 0x7f44cc07e330 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:07.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.100+0000 7f44d3c51700 1 -- 172.21.15.49:0/3877939832 >> 172.21.15.49:0/3877939832 conn(0x7f44cc103500 msgr2=0x7f44cc104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:07.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.100+0000 7f44d3c51700 1 -- 172.21.15.49:0/3877939832 shutdown_connections 2024-03-26T17:52:07.591 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:07.100+0000 7f44d3c51700 1 -- 172.21.15.49:0/3877939832 wait complete. 2024-03-26T17:52:07.591 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling cephadm module... 2024-03-26T17:52:08.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:08 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2488410281' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-26T17:52:08.649 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:08 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: ignoring --setuser ceph since I am not root 2024-03-26T17:52:08.649 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:08 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: ignoring --setgroup ceph since I am not root 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.003+0000 7f15eaba1700 1 Processor -- start 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.003+0000 7f15eaba1700 1 -- start start 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.003+0000 7f15eaba1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e407caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.003+0000 7f15eaba1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15e407cfe0 con 0x7f15e407e5f0 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.004+0000 7f15e9b9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e407caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.004+0000 7f15e9b9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e407caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48396/0 (socket says 172.21.15.49:48396) 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.004+0000 7f15e9b9f700 1 -- 172.21.15.49:0/1118367281 learned_addr learned my addr 172.21.15.49:0/1118367281 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.004+0000 7f15e9b9f700 1 -- 172.21.15.49:0/1118367281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e407d160 con 0x7f15e407e5f0 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.004+0000 7f15e9b9f700 1 --2- 172.21.15.49:0/1118367281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e407caa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f15e00098a0 tx=0x7f15e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b428a716e3ec6f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.004+0000 7f15e8b9d700 1 -- 172.21.15.49:0/1118367281 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15e0004160 con 0x7f15e407e5f0 2024-03-26T17:52:08.899 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.004+0000 7f15e8b9d700 1 -- 172.21.15.49:0/1118367281 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f15e0004300 con 0x7f15e407e5f0 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.004+0000 7f15e8b9d700 1 -- 172.21.15.49:0/1118367281 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15e0011b50 con 0x7f15e407e5f0 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.005+0000 7f15eaba1700 1 -- 172.21.15.49:0/1118367281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 msgr2=0x7f15e407caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.005+0000 7f15eaba1700 1 --2- 172.21.15.49:0/1118367281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e407caa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f15e00098a0 tx=0x7f15e0009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.005+0000 7f15eaba1700 1 -- 172.21.15.49:0/1118367281 shutdown_connections 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.005+0000 7f15eaba1700 1 --2- 172.21.15.49:0/1118367281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e407caa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.005+0000 7f15eaba1700 1 -- 172.21.15.49:0/1118367281 >> 172.21.15.49:0/1118367281 conn(0x7f15e4103390 msgr2=0x7f15e41057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.005+0000 7f15eaba1700 1 -- 172.21.15.49:0/1118367281 shutdown_connections 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.005+0000 7f15eaba1700 1 -- 172.21.15.49:0/1118367281 wait complete. 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15eaba1700 1 Processor -- start 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15eaba1700 1 -- start start 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15eaba1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:08.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15eaba1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15e410e360 con 0x7f15e407e5f0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15e9b9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15e9b9f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48412/0 (socket says 172.21.15.49:48412) 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15e9b9f700 1 -- 172.21.15.49:0/2488410281 learned_addr learned my addr 172.21.15.49:0/2488410281 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15e9b9f700 1 -- 172.21.15.49:0/2488410281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e419e480 con 0x7f15e407e5f0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15e9b9f700 1 --2- 172.21.15.49:0/2488410281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e419df40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f15e0011b60 tx=0x7f15e0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15daffd700 1 -- 172.21.15.49:0/2488410281 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15e0003970 con 0x7f15e407e5f0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15daffd700 1 -- 172.21.15.49:0/2488410281 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f15e0003b10 con 0x7f15e407e5f0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15eaba1700 1 -- 172.21.15.49:0/2488410281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e419e710 con 0x7f15e407e5f0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.006+0000 7f15daffd700 1 -- 172.21.15.49:0/2488410281 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15e0024610 con 0x7f15e407e5f0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.007+0000 7f15eaba1700 1 -- 172.21.15.49:0/2488410281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e41a1400 con 0x7f15e407e5f0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.008+0000 7f15daffd700 1 -- 172.21.15.49:0/2488410281 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 46283+0+0 (secure 0 0 0) 0x7f15e001a430 con 0x7f15e407e5f0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.008+0000 7f15eaba1700 1 -- 172.21.15.49:0/2488410281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e40694b0 con 0x7f15e407e5f0 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.008+0000 7f15daffd700 1 --2- 172.21.15.49:0/2488410281 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f15d0039180 0x7f15d003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:08.901 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.008+0000 7f15daffd700 1 -- 172.21.15.49:0/2488410281 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f15e0011e00 con 0x7f15e407e5f0 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.008+0000 7f15e939e700 1 --2- 172.21.15.49:0/2488410281 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f15d0039180 0x7f15d003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.008+0000 7f15e939e700 1 --2- 172.21.15.49:0/2488410281 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f15d0039180 0x7f15d003b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15d4006fd0 tx=0x7f15d4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.013+0000 7f15daffd700 1 -- 172.21.15.49:0/2488410281 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f15e0022db0 con 0x7f15e407e5f0 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.274+0000 7f15eaba1700 1 -- 172.21.15.49:0/2488410281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f15e41a15d0 con 0x7f15e407e5f0 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.325+0000 7f15daffd700 1 -- 172.21.15.49:0/2488410281 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f15e001a9f0 con 0x7f15e407e5f0 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.327+0000 7f15daffd700 1 -- 172.21.15.49:0/2488410281 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 5) ==== 46294+0+0 (secure 0 0 0) 0x7f15e0016ca0 con 0x7f15e407e5f0 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.328+0000 7f15eaba1700 1 -- 172.21.15.49:0/2488410281 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f15d0039180 msgr2=0x7f15d003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.328+0000 7f15eaba1700 1 --2- 172.21.15.49:0/2488410281 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f15d0039180 0x7f15d003b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15d4006fd0 tx=0x7f15d4006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.328+0000 7f15eaba1700 1 -- 172.21.15.49:0/2488410281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 msgr2=0x7f15e419df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.328+0000 7f15eaba1700 1 --2- 172.21.15.49:0/2488410281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e419df40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f15e0011b60 tx=0x7f15e0011c40 comp rx=0 tx=0).stop 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.328+0000 7f15eaba1700 1 -- 172.21.15.49:0/2488410281 shutdown_connections 2024-03-26T17:52:08.902 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.328+0000 7f15eaba1700 1 --2- 172.21.15.49:0/2488410281 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f15d0039180 0x7f15d003b640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:08.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.328+0000 7f15eaba1700 1 --2- 172.21.15.49:0/2488410281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e407e5f0 0x7f15e419df40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:08.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.328+0000 7f15eaba1700 1 -- 172.21.15.49:0/2488410281 >> 172.21.15.49:0/2488410281 conn(0x7f15e4103390 msgr2=0x7f15e4103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:08.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.328+0000 7f15eaba1700 1 -- 172.21.15.49:0/2488410281 shutdown_connections 2024-03-26T17:52:08.903 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:08.328+0000 7f15eaba1700 1 -- 172.21.15.49:0/2488410281 wait complete. 2024-03-26T17:52:08.994 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:08 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:08.902+0000 7f06a8368200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T17:52:08.994 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:09 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:09.003+0000 7f06a8368200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T17:52:09.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:09 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2488410281' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-26T17:52:09.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:09 smithi049 ceph-mon[32331]: mgrmap e5: x(active, since 11s) 2024-03-26T17:52:09.929 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:09 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:09.691+0000 7f06a8368200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T17:52:09.929 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:09 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:09.789+0000 7f06a8368200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T17:52:09.929 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:09 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:09.936+0000 7f06a8368200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T17:52:10.214 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:10.036+0000 7f06a8368200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T17:52:10.214 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:10.122+0000 7f06a8368200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T17:52:10.222 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:52:10.222 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.371+0000 7f93c919d700 1 Processor -- start 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.371+0000 7f93c919d700 1 -- start start 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.372+0000 7f93c919d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c410aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.372+0000 7f93c919d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f93c407c620 con 0x7f93c41087b0 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.372+0000 7f93c3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c410aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.372+0000 7f93c3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c410aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58826/0 (socket says 172.21.15.49:58826) 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.372+0000 7f93c3fff700 1 -- 172.21.15.49:0/2395045637 learned_addr learned my addr 172.21.15.49:0/2395045637 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.372+0000 7f93c3fff700 1 -- 172.21.15.49:0/2395045637 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c410b0e0 con 0x7f93c41087b0 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.372+0000 7f93c3fff700 1 --2- 172.21.15.49:0/2395045637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c410aba0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f93b40098a0 tx=0x7f93b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8ad916cd3aeb2c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.372+0000 7f93c2ffd700 1 -- 172.21.15.49:0/2395045637 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93b4004160 con 0x7f93c41087b0 2024-03-26T17:52:10.223 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.372+0000 7f93c2ffd700 1 -- 172.21.15.49:0/2395045637 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93b4004300 con 0x7f93c41087b0 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.372+0000 7f93c2ffd700 1 -- 172.21.15.49:0/2395045637 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93b4011b50 con 0x7f93c41087b0 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.373+0000 7f93c919d700 1 -- 172.21.15.49:0/2395045637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 msgr2=0x7f93c410aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.373+0000 7f93c919d700 1 --2- 172.21.15.49:0/2395045637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c410aba0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f93b40098a0 tx=0x7f93b4009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.373+0000 7f93c919d700 1 -- 172.21.15.49:0/2395045637 shutdown_connections 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.373+0000 7f93c919d700 1 --2- 172.21.15.49:0/2395045637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c410aba0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.373+0000 7f93c919d700 1 -- 172.21.15.49:0/2395045637 >> 172.21.15.49:0/2395045637 conn(0x7f93c4102240 msgr2=0x7f93c4104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.373+0000 7f93c919d700 1 -- 172.21.15.49:0/2395045637 shutdown_connections 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.373+0000 7f93c919d700 1 -- 172.21.15.49:0/2395045637 wait complete. 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.374+0000 7f93c919d700 1 Processor -- start 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.374+0000 7f93c919d700 1 -- start start 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c919d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c919d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f93c410c100 con 0x7f93c41087b0 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c419bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58832/0 (socket says 172.21.15.49:58832) 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c3fff700 1 -- 172.21.15.49:0/4092105369 learned_addr learned my addr 172.21.15.49:0/4092105369 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c3fff700 1 -- 172.21.15.49:0/4092105369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c419c230 con 0x7f93c41087b0 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c3fff700 1 --2- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c419bcf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009fb0 tx=0x7f93b4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c17fa700 1 -- 172.21.15.49:0/4092105369 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93b40038b0 con 0x7f93c41087b0 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c919d700 1 -- 172.21.15.49:0/4092105369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c419c4c0 con 0x7f93c41087b0 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c17fa700 1 -- 172.21.15.49:0/4092105369 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93b4011e50 con 0x7f93c41087b0 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.375+0000 7f93c17fa700 1 -- 172.21.15.49:0/4092105369 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93b4019420 con 0x7f93c41087b0 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.376+0000 7f93c919d700 1 -- 172.21.15.49:0/4092105369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c419f1a0 con 0x7f93c41087b0 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.376+0000 7f93c17fa700 1 -- 172.21.15.49:0/4092105369 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 46294+0+0 (secure 0 0 0) 0x7f93b4019720 con 0x7f93c41087b0 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.377+0000 7f93c17fa700 1 --2- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f93ac039130 0x7f93ac03b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.377+0000 7f93c919d700 1 -- 172.21.15.49:0/4092105369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c40694b0 con 0x7f93c41087b0 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.377+0000 7f93c17fa700 1 -- 172.21.15.49:0/4092105369 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f93b404e480 con 0x7f93c41087b0 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.377+0000 7f93c37fe700 1 -- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f93ac039130 msgr2=0x7f93ac03b5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/48778628 2024-03-26T17:52:10.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.377+0000 7f93c37fe700 1 --2- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f93ac039130 0x7f93ac03b5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.386+0000 7f93c17fa700 1 -- 172.21.15.49:0/4092105369 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f93b4020240 con 0x7f93c41087b0 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.578+0000 7f93c37fe700 1 -- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f93ac039130 msgr2=0x7f93ac03b5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/48778628 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.578+0000 7f93c37fe700 1 --2- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f93ac039130 0x7f93ac03b5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.652+0000 7f93c919d700 1 -- 172.21.15.49:0/4092105369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f93c419f370 con 0x7f93c41087b0 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.652+0000 7f93c17fa700 1 -- 172.21.15.49:0/4092105369 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f93b400bc70 con 0x7f93c41087b0 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.655+0000 7f93c919d700 1 -- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f93ac039130 msgr2=0x7f93ac03b5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.655+0000 7f93c919d700 1 --2- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f93ac039130 0x7f93ac03b5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.655+0000 7f93c919d700 1 -- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 msgr2=0x7f93c419bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.655+0000 7f93c919d700 1 --2- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c419bcf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009fb0 tx=0x7f93b4011bf0 comp rx=0 tx=0).stop 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.655+0000 7f93c919d700 1 -- 172.21.15.49:0/4092105369 shutdown_connections 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.655+0000 7f93c919d700 1 --2- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f93ac039130 0x7f93ac03b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.655+0000 7f93c919d700 1 --2- 172.21.15.49:0/4092105369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93c41087b0 0x7f93c419bcf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:10.226 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.655+0000 7f93c919d700 1 -- 172.21.15.49:0/4092105369 >> 172.21.15.49:0/4092105369 conn(0x7f93c4102240 msgr2=0x7f93c4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:10.227 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.655+0000 7f93c919d700 1 -- 172.21.15.49:0/4092105369 shutdown_connections 2024-03-26T17:52:10.227 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:09.655+0000 7f93c919d700 1 -- 172.21.15.49:0/4092105369 wait complete. 2024-03-26T17:52:10.227 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-03-26T17:52:10.227 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 5... 2024-03-26T17:52:10.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:10 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4092105369' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T17:52:10.500 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:10.400+0000 7f06a8368200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T17:52:10.761 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:10.517+0000 7f06a8368200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T17:52:10.761 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:10.608+0000 7f06a8368200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T17:52:10.761 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:10.708+0000 7f06a8368200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T17:52:11.210 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:10.890+0000 7f06a8368200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T17:52:12.261 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:11 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:11.900+0000 7f06a8368200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T17:52:12.261 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:12.036+0000 7f06a8368200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T17:52:12.668 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:12.382+0000 7f06a8368200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T17:52:12.668 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:12.474+0000 7f06a8368200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T17:52:12.668 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:12.676+0000 7f06a8368200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:52:13.011 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:12.867+0000 7f06a8368200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T17:52:13.422 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:13 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:13.162+0000 7f06a8368200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:52:13.422 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:13 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:13.430+0000 7f06a8368200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T17:52:13.761 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:13 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:13.676+0000 7f06a8368200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T17:52:14.761 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:14 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:14.512+0000 7f06a8368200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T17:52:15.425 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:15.107+0000 7f06a8368200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T17:52:15.426 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:15.208+0000 7f06a8368200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T17:52:15.426 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:15.433+0000 7f06a8368200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T17:52:15.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: Active manager daemon x restarted 2024-03-26T17:52:15.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: Activating manager daemon x 2024-03-26T17:52:15.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: osdmap e2: 0 total, 0 up, 0 in 2024-03-26T17:52:15.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: mgrmap e6: x(active, starting, since 0.00352412s) 2024-03-26T17:52:15.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:52:15.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T17:52:15.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T17:52:15.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T17:52:15.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T17:52:15.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: Manager daemon x is now available 2024-03-26T17:52:15.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:15.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:15.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:15 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:16.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:16 smithi049 ceph-mon[32331]: Found migration_current of "None". Setting to last migration. 2024-03-26T17:52:16.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:16 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:16.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:16 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T17:52:16.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:16 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T17:52:16.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:16 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:16.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:16 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:16.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:16 smithi049 ceph-mon[32331]: [26/Mar/2024:17:52:16] ENGINE Bus STARTING 2024-03-26T17:52:16.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:16 smithi049 ceph-mon[32331]: mgrmap e7: x(active, since 1.00532s) 2024-03-26T17:52:17.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:52:17.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-26T17:52:17.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T17:52:17.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:52:17.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.611+0000 7f77400c4700 1 Processor -- start 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.611+0000 7f77400c4700 1 -- start start 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.611+0000 7f77400c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f7738106c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.611+0000 7f77400c4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f773807c5a0 con 0x7f7738106820 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.611+0000 7f773de60700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f7738106c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.611+0000 7f773de60700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f7738106c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58844/0 (socket says 172.21.15.49:58844) 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.611+0000 7f773de60700 1 -- 172.21.15.49:0/2331808766 learned_addr learned my addr 172.21.15.49:0/2331808766 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.612+0000 7f773de60700 1 -- 172.21.15.49:0/2331808766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7738107180 con 0x7f7738106820 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.612+0000 7f773de60700 1 --2- 172.21.15.49:0/2331808766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f7738106c40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f77340098a0 tx=0x7f7734009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb12a2c51cb5886 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.612+0000 7f773ce5e700 1 -- 172.21.15.49:0/2331808766 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7734004160 con 0x7f7738106820 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.612+0000 7f773ce5e700 1 -- 172.21.15.49:0/2331808766 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7734004300 con 0x7f7738106820 2024-03-26T17:52:17.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.612+0000 7f773ce5e700 1 -- 172.21.15.49:0/2331808766 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7734011b50 con 0x7f7738106820 2024-03-26T17:52:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.613+0000 7f77400c4700 1 -- 172.21.15.49:0/2331808766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 msgr2=0x7f7738106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.613+0000 7f77400c4700 1 --2- 172.21.15.49:0/2331808766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f7738106c40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f77340098a0 tx=0x7f7734009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.613+0000 7f77400c4700 1 -- 172.21.15.49:0/2331808766 shutdown_connections 2024-03-26T17:52:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.613+0000 7f77400c4700 1 --2- 172.21.15.49:0/2331808766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f7738106c40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.613+0000 7f77400c4700 1 -- 172.21.15.49:0/2331808766 >> 172.21.15.49:0/2331808766 conn(0x7f7738102400 msgr2=0x7f7738104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.613+0000 7f77400c4700 1 -- 172.21.15.49:0/2331808766 shutdown_connections 2024-03-26T17:52:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.613+0000 7f77400c4700 1 -- 172.21.15.49:0/2331808766 wait complete. 2024-03-26T17:52:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.613+0000 7f77400c4700 1 Processor -- start 2024-03-26T17:52:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f77400c4700 1 -- start start 2024-03-26T17:52:17.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f77400c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f773819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f77400c4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7738108090 con 0x7f7738106820 2024-03-26T17:52:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f773de60700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f773819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f773de60700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f773819e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58846/0 (socket says 172.21.15.49:58846) 2024-03-26T17:52:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f773de60700 1 -- 172.21.15.49:0/2951887268 learned_addr learned my addr 172.21.15.49:0/2951887268 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f773de60700 1 -- 172.21.15.49:0/2951887268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773819e5f0 con 0x7f7738106820 2024-03-26T17:52:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f773de60700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f773819e0b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7734009fb0 tx=0x7f7734011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77340038b0 con 0x7f7738106820 2024-03-26T17:52:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f77400c4700 1 -- 172.21.15.49:0/2951887268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773819e880 con 0x7f7738106820 2024-03-26T17:52:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7734011e50 con 0x7f7738106820 2024-03-26T17:52:17.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.614+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7734019420 con 0x7f7738106820 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.615+0000 7f77400c4700 1 -- 172.21.15.49:0/2951887268 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77381a1570 con 0x7f7738106820 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.615+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 46294+0+0 (secure 0 0 0) 0x7f7734019730 con 0x7f7738106820 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.616+0000 7f772effd700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 0x7f772403b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.616+0000 7f77400c4700 1 -- 172.21.15.49:0/2951887268 --> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f77380694b0 con 0x7f7724039180 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.616+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f773404d3e0 con 0x7f7738106820 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.616+0000 7f773d65f700 1 -- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 msgr2=0x7f772403b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/48778628 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.616+0000 7f773d65f700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 0x7f772403b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.817+0000 7f773d65f700 1 -- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 msgr2=0x7f772403b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/48778628 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:10.817+0000 7f773d65f700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 0x7f772403b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:11.218+0000 7f773d65f700 1 -- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 msgr2=0x7f772403b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/48778628 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:11.218+0000 7f773d65f700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 0x7f772403b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T17:52:17.028 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:12.019+0000 7f773d65f700 1 -- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 msgr2=0x7f772403b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/48778628 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:12.019+0000 7f773d65f700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 0x7f772403b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:13.620+0000 7f773d65f700 1 -- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 msgr2=0x7f772403b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/48778628 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:13.620+0000 7f773d65f700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 0x7f772403b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:15.449+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 6) ==== 46061+0+0 (secure 0 0 0) 0x7f7734003a50 con 0x7f7738106820 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:15.449+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 msgr2=0x7f772403b640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:15.449+0000 7f772effd700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 0x7f772403b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.450+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 7) ==== 46188+0+0 (secure 0 0 0) 0x7f773404ed40 con 0x7f7738106820 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.450+0000 7f772effd700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f772403c880 0x7f772403ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.450+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 --> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7734012ea0 con 0x7f772403c880 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.451+0000 7f773d65f700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f772403c880 0x7f772403ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:17.029 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.451+0000 7f773d65f700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f772403c880 0x7f772403ec70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7728003a10 tx=0x7f77280092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.452+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 <== mgr.14120 v2:172.21.15.49:6800/4249557516 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7734012ea0 con 0x7f772403c880 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.458+0000 7f77400c4700 1 -- 172.21.15.49:0/2951887268 --> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f77380627f0 con 0x7f772403c880 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f772effd700 1 -- 172.21.15.49:0/2951887268 <== mgr.14120 v2:172.21.15.49:6800/4249557516 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f77380627f0 con 0x7f772403c880 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 -- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f772403c880 msgr2=0x7f772403ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f772403c880 0x7f772403ec70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7728003a10 tx=0x7f77280092b0 comp rx=0 tx=0).stop 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 -- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 msgr2=0x7f773819e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f773819e0b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7734009fb0 tx=0x7f7734011bf0 comp rx=0 tx=0).stop 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 -- 172.21.15.49:0/2951887268 shutdown_connections 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f772403c880 0x7f772403ec70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7738106820 0x7f773819e0b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:17.030 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 --2- 172.21.15.49:0/2951887268 >> [v2:172.21.15.49:6800/48778628,v1:172.21.15.49:6801/48778628] conn(0x7f7724039180 0x7f772403b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:17.031 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 -- 172.21.15.49:0/2951887268 >> 172.21.15.49:0/2951887268 conn(0x7f7738102400 msgr2=0x7f7738194cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:17.031 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 -- 172.21.15.49:0/2951887268 shutdown_connections 2024-03-26T17:52:17.031 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:16.459+0000 7f77400c4700 1 -- 172.21.15.49:0/2951887268 wait complete. 2024-03-26T17:52:17.031 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 5 is available 2024-03-26T17:52:17.031 INFO:teuthology.orchestra.run.smithi049.stdout:Setting orchestrator backend to cephadm... 2024-03-26T17:52:17.675 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:17 smithi049 ceph-mon[32331]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T17:52:17.675 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:17 smithi049 ceph-mon[32331]: [26/Mar/2024:17:52:16] ENGINE Serving on http://172.21.15.49:8765 2024-03-26T17:52:17.675 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:17 smithi049 ceph-mon[32331]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T17:52:17.676 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:17 smithi049 ceph-mon[32331]: [26/Mar/2024:17:52:16] ENGINE Serving on https://172.21.15.49:7150 2024-03-26T17:52:17.676 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:17 smithi049 ceph-mon[32331]: [26/Mar/2024:17:52:16] ENGINE Bus STARTED 2024-03-26T17:52:17.676 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:17 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:18.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.448+0000 7f253c733700 1 Processor -- start 2024-03-26T17:52:18.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.448+0000 7f253c733700 1 -- start start 2024-03-26T17:52:18.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.449+0000 7f253c733700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f2534109890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.449+0000 7f253c733700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2534109dd0 con 0x7f2534109470 2024-03-26T17:52:18.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.449+0000 7f253a4cf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f2534109890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.449+0000 7f253a4cf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f2534109890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58924/0 (socket says 172.21.15.49:58924) 2024-03-26T17:52:18.285 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.449+0000 7f253a4cf700 1 -- 172.21.15.49:0/2048024227 learned_addr learned my addr 172.21.15.49:0/2048024227 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.449+0000 7f253a4cf700 1 -- 172.21.15.49:0/2048024227 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2534107c90 con 0x7f2534109470 2024-03-26T17:52:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.449+0000 7f253a4cf700 1 --2- 172.21.15.49:0/2048024227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f2534109890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f25240098a0 tx=0x7f2524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c83cf38f125d971 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.449+0000 7f25394cd700 1 -- 172.21.15.49:0/2048024227 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2524004160 con 0x7f2534109470 2024-03-26T17:52:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.449+0000 7f25394cd700 1 -- 172.21.15.49:0/2048024227 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2524004300 con 0x7f2534109470 2024-03-26T17:52:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.449+0000 7f25394cd700 1 -- 172.21.15.49:0/2048024227 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2524011b50 con 0x7f2534109470 2024-03-26T17:52:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.450+0000 7f253c733700 1 -- 172.21.15.49:0/2048024227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 msgr2=0x7f2534109890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.450+0000 7f253c733700 1 --2- 172.21.15.49:0/2048024227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f2534109890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f25240098a0 tx=0x7f2524009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.450+0000 7f253c733700 1 -- 172.21.15.49:0/2048024227 shutdown_connections 2024-03-26T17:52:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.450+0000 7f253c733700 1 --2- 172.21.15.49:0/2048024227 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f2534109890 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.286 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.450+0000 7f253c733700 1 -- 172.21.15.49:0/2048024227 >> 172.21.15.49:0/2048024227 conn(0x7f2534103630 msgr2=0x7f2534105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.450+0000 7f253c733700 1 -- 172.21.15.49:0/2048024227 shutdown_connections 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.450+0000 7f253c733700 1 -- 172.21.15.49:0/2048024227 wait complete. 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.451+0000 7f253c733700 1 Processor -- start 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.451+0000 7f253c733700 1 -- start start 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.451+0000 7f253c733700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f253419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.451+0000 7f253c733700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f253410c3b0 con 0x7f2534109470 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.451+0000 7f253a4cf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f253419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.451+0000 7f253a4cf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f253419be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58940/0 (socket says 172.21.15.49:58940) 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.451+0000 7f253a4cf700 1 -- 172.21.15.49:0/1246560120 learned_addr learned my addr 172.21.15.49:0/1246560120 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.451+0000 7f253a4cf700 1 -- 172.21.15.49:0/1246560120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253419c3d0 con 0x7f2534109470 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.451+0000 7f253a4cf700 1 --2- 172.21.15.49:0/1246560120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f253419be90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2524011670 tx=0x7f2524011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.451+0000 7f252b7fe700 1 -- 172.21.15.49:0/1246560120 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25240039c0 con 0x7f2534109470 2024-03-26T17:52:18.287 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.452+0000 7f253c733700 1 -- 172.21.15.49:0/1246560120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253419c660 con 0x7f2534109470 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.452+0000 7f252b7fe700 1 -- 172.21.15.49:0/1246560120 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2524011e50 con 0x7f2534109470 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.452+0000 7f252b7fe700 1 -- 172.21.15.49:0/1246560120 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f252401a5c0 con 0x7f2534109470 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.452+0000 7f253c733700 1 -- 172.21.15.49:0/1246560120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253419f350 con 0x7f2534109470 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.453+0000 7f252b7fe700 1 -- 172.21.15.49:0/1246560120 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 7) ==== 46188+0+0 (secure 0 0 0) 0x7f2524024430 con 0x7f2534109470 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.453+0000 7f253c733700 1 -- 172.21.15.49:0/1246560120 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f253410c530 con 0x7f2534109470 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.453+0000 7f252b7fe700 1 --2- 172.21.15.49:0/1246560120 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f2520039030 0x7f252003b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.453+0000 7f252b7fe700 1 -- 172.21.15.49:0/1246560120 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f252404d480 con 0x7f2534109470 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.453+0000 7f2539cce700 1 --2- 172.21.15.49:0/1246560120 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f2520039030 0x7f252003b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.453+0000 7f2539cce700 1 --2- 172.21.15.49:0/1246560120 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f2520039030 0x7f252003b4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2530006fd0 tx=0x7f2530006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.458+0000 7f252b7fe700 1 -- 172.21.15.49:0/1246560120 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f252402d910 con 0x7f2534109470 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.501+0000 7f252b7fe700 1 -- 172.21.15.49:0/1246560120 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f252402d3f0 con 0x7f2534109470 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.683+0000 7f253c733700 1 -- 172.21.15.49:0/1246560120 --> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f25341047b0 con 0x7f2520039030 2024-03-26T17:52:18.288 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.689+0000 7f252b7fe700 1 -- 172.21.15.49:0/1246560120 <== mgr.14120 v2:172.21.15.49:6800/4249557516 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25341047b0 con 0x7f2520039030 2024-03-26T17:52:18.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.691+0000 7f253c733700 1 -- 172.21.15.49:0/1246560120 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f2520039030 msgr2=0x7f252003b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:18.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.691+0000 7f253c733700 1 --2- 172.21.15.49:0/1246560120 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f2520039030 0x7f252003b4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2530006fd0 tx=0x7f2530006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:18.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.691+0000 7f253c733700 1 -- 172.21.15.49:0/1246560120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 msgr2=0x7f253419be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:18.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.691+0000 7f253c733700 1 --2- 172.21.15.49:0/1246560120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f253419be90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2524011670 tx=0x7f2524011b60 comp rx=0 tx=0).stop 2024-03-26T17:52:18.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.691+0000 7f253c733700 1 -- 172.21.15.49:0/1246560120 shutdown_connections 2024-03-26T17:52:18.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.691+0000 7f253c733700 1 --2- 172.21.15.49:0/1246560120 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f2520039030 0x7f252003b4f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.691+0000 7f253c733700 1 --2- 172.21.15.49:0/1246560120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2534109470 0x7f253419be90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.691+0000 7f253c733700 1 -- 172.21.15.49:0/1246560120 >> 172.21.15.49:0/1246560120 conn(0x7f2534103630 msgr2=0x7f25341040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:18.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.691+0000 7f253c733700 1 -- 172.21.15.49:0/1246560120 shutdown_connections 2024-03-26T17:52:18.289 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:17.691+0000 7f253c733700 1 -- 172.21.15.49:0/1246560120 wait complete. 2024-03-26T17:52:18.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:18 smithi049 ceph-mon[32331]: mgrmap e8: x(active, since 2s) 2024-03-26T17:52:18.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:18 smithi049 ceph-mon[32331]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:52:18.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:18 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:18.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:18 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:19.533 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:19 smithi049 ceph-mon[32331]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.707+0000 7f008a933700 1 Processor -- start 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.707+0000 7f008a933700 1 -- start start 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.707+0000 7f008a933700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f00841083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.707+0000 7f008a933700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0084108930 con 0x7f0084107fd0 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.708+0000 7f0083fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f00841083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.708+0000 7f0083fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f00841083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44078/0 (socket says 172.21.15.49:44078) 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.708+0000 7f0083fff700 1 -- 172.21.15.49:0/1856700319 learned_addr learned my addr 172.21.15.49:0/1856700319 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.708+0000 7f0083fff700 1 -- 172.21.15.49:0/1856700319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0084109090 con 0x7f0084107fd0 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.708+0000 7f0083fff700 1 --2- 172.21.15.49:0/1856700319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f00841083f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f00780098a0 tx=0x7f0078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5574a52c643ee29e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.708+0000 7f0082ffd700 1 -- 172.21.15.49:0/1856700319 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0078004160 con 0x7f0084107fd0 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.708+0000 7f0082ffd700 1 -- 172.21.15.49:0/1856700319 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0078004300 con 0x7f0084107fd0 2024-03-26T17:52:19.541 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.708+0000 7f0082ffd700 1 -- 172.21.15.49:0/1856700319 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0078011b50 con 0x7f0084107fd0 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.709+0000 7f008a933700 1 -- 172.21.15.49:0/1856700319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 msgr2=0x7f00841083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.709+0000 7f008a933700 1 --2- 172.21.15.49:0/1856700319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f00841083f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f00780098a0 tx=0x7f0078009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.709+0000 7f008a933700 1 -- 172.21.15.49:0/1856700319 shutdown_connections 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.709+0000 7f008a933700 1 --2- 172.21.15.49:0/1856700319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f00841083f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.709+0000 7f008a933700 1 -- 172.21.15.49:0/1856700319 >> 172.21.15.49:0/1856700319 conn(0x7f0084103570 msgr2=0x7f00841059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.709+0000 7f008a933700 1 -- 172.21.15.49:0/1856700319 shutdown_connections 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.709+0000 7f008a933700 1 -- 172.21.15.49:0/1856700319 wait complete. 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.709+0000 7f008a933700 1 Processor -- start 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.710+0000 7f008a933700 1 -- start start 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.710+0000 7f008a933700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f008419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.710+0000 7f008a933700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f008410c4d0 con 0x7f0084107fd0 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.710+0000 7f0083fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f008419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:19.542 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.710+0000 7f0083fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f008419bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44084/0 (socket says 172.21.15.49:44084) 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.710+0000 7f0083fff700 1 -- 172.21.15.49:0/3201468642 learned_addr learned my addr 172.21.15.49:0/3201468642 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.710+0000 7f0083fff700 1 -- 172.21.15.49:0/3201468642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008419c450 con 0x7f0084107fd0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.710+0000 7f0083fff700 1 --2- 172.21.15.49:0/3201468642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f008419bf10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0078011b60 tx=0x7f0078011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.711+0000 7f00817fa700 1 -- 172.21.15.49:0/3201468642 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0078003970 con 0x7f0084107fd0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.711+0000 7f008a933700 1 -- 172.21.15.49:0/3201468642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008419c6e0 con 0x7f0084107fd0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.711+0000 7f00817fa700 1 -- 172.21.15.49:0/3201468642 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0078003b10 con 0x7f0084107fd0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.711+0000 7f00817fa700 1 -- 172.21.15.49:0/3201468642 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0078024610 con 0x7f0084107fd0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.711+0000 7f008a933700 1 -- 172.21.15.49:0/3201468642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f008419f3d0 con 0x7f0084107fd0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.712+0000 7f00817fa700 1 -- 172.21.15.49:0/3201468642 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f007801a9f0 con 0x7f0084107fd0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.712+0000 7f008a933700 1 -- 172.21.15.49:0/3201468642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00840694b0 con 0x7f0084107fd0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.712+0000 7f00817fa700 1 --2- 172.21.15.49:0/3201468642 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f006c039180 0x7f006c03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.712+0000 7f00817fa700 1 -- 172.21.15.49:0/3201468642 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0078011e00 con 0x7f0084107fd0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.712+0000 7f00837fe700 1 --2- 172.21.15.49:0/3201468642 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f006c039180 0x7f006c03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.713+0000 7f00837fe700 1 --2- 172.21.15.49:0/3201468642 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f006c039180 0x7f006c03b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0074006fd0 tx=0x7f0074006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:19.543 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.717+0000 7f00817fa700 1 -- 172.21.15.49:0/3201468642 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f007800b3e0 con 0x7f0084107fd0 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.920+0000 7f008a933700 1 -- 172.21.15.49:0/3201468642 --> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f00841047e0 con 0x7f006c039180 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.921+0000 7f00817fa700 1 -- 172.21.15.49:0/3201468642 <== mgr.14120 v2:172.21.15.49:6800/4249557516 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f00841047e0 con 0x7f006c039180 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.923+0000 7f008a933700 1 -- 172.21.15.49:0/3201468642 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f006c039180 msgr2=0x7f006c03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.923+0000 7f008a933700 1 --2- 172.21.15.49:0/3201468642 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f006c039180 0x7f006c03b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0074006fd0 tx=0x7f0074006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.923+0000 7f008a933700 1 -- 172.21.15.49:0/3201468642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 msgr2=0x7f008419bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.923+0000 7f008a933700 1 --2- 172.21.15.49:0/3201468642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f008419bf10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0078011b60 tx=0x7f0078011c40 comp rx=0 tx=0).stop 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.924+0000 7f008a933700 1 -- 172.21.15.49:0/3201468642 shutdown_connections 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.924+0000 7f008a933700 1 --2- 172.21.15.49:0/3201468642 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f006c039180 0x7f006c03b640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.924+0000 7f008a933700 1 --2- 172.21.15.49:0/3201468642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0084107fd0 0x7f008419bf10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.924+0000 7f008a933700 1 -- 172.21.15.49:0/3201468642 >> 172.21.15.49:0/3201468642 conn(0x7f0084103570 msgr2=0x7f00841040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.924+0000 7f008a933700 1 -- 172.21.15.49:0/3201468642 shutdown_connections 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:18.924+0000 7f008a933700 1 -- 172.21.15.49:0/3201468642 wait complete. 2024-03-26T17:52:19.544 INFO:teuthology.orchestra.run.smithi049.stdout:Generating ssh key... 2024-03-26T17:52:20.511 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: Generating public/private rsa key pair. 2024-03-26T17:52:20.511 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: Your identification has been saved in /tmp/tmpn35upore/key. 2024-03-26T17:52:20.511 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: Your public key has been saved in /tmp/tmpn35upore/key.pub. 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: The key fingerprint is: 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: SHA256:aTesMEaLE5s7NCDhcdIddkLvOqug7S//PgTrRN7Nk7Y ceph-5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: The key's randomart image is: 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: +---[RSA 3072]----+ 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: |.o..o=.. | 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: |..+...+ | 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: |... . .. | 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: | . .o*.. o | 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: | oB+=+S.+ | 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: | .+=++*o . | 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: | . oo+ ..o | 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: |..o ..+ E | 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: |..o=++o. | 2024-03-26T17:52:20.512 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: +----[SHA256]-----+ 2024-03-26T17:52:20.772 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.958+0000 7f38aeff1700 1 Processor -- start 2024-03-26T17:52:20.772 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.958+0000 7f38aeff1700 1 -- start start 2024-03-26T17:52:20.772 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.958+0000 7f38aeff1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a8108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:20.772 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.959+0000 7f38aeff1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81088d0 con 0x7f38a8107f70 2024-03-26T17:52:20.772 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.959+0000 7f38acd8d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a8108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.959+0000 7f38acd8d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a8108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44088/0 (socket says 172.21.15.49:44088) 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.959+0000 7f38acd8d700 1 -- 172.21.15.49:0/3334205285 learned_addr learned my addr 172.21.15.49:0/3334205285 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.959+0000 7f38acd8d700 1 -- 172.21.15.49:0/3334205285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a8109080 con 0x7f38a8107f70 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.959+0000 7f38acd8d700 1 --2- 172.21.15.49:0/3334205285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a8108390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f389c0098a0 tx=0x7f389c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1c26eede135163 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.959+0000 7f38a77fe700 1 -- 172.21.15.49:0/3334205285 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f389c004160 con 0x7f38a8107f70 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.959+0000 7f38a77fe700 1 -- 172.21.15.49:0/3334205285 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f389c004300 con 0x7f38a8107f70 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.959+0000 7f38a77fe700 1 -- 172.21.15.49:0/3334205285 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f389c011b50 con 0x7f38a8107f70 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.960+0000 7f38aeff1700 1 -- 172.21.15.49:0/3334205285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 msgr2=0x7f38a8108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.960+0000 7f38aeff1700 1 --2- 172.21.15.49:0/3334205285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a8108390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f389c0098a0 tx=0x7f389c009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.960+0000 7f38aeff1700 1 -- 172.21.15.49:0/3334205285 shutdown_connections 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.960+0000 7f38aeff1700 1 --2- 172.21.15.49:0/3334205285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a8108390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.960+0000 7f38aeff1700 1 -- 172.21.15.49:0/3334205285 >> 172.21.15.49:0/3334205285 conn(0x7f38a8103510 msgr2=0x7f38a8105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:20.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.960+0000 7f38aeff1700 1 -- 172.21.15.49:0/3334205285 shutdown_connections 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.960+0000 7f38aeff1700 1 -- 172.21.15.49:0/3334205285 wait complete. 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.961+0000 7f38aeff1700 1 Processor -- start 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.961+0000 7f38aeff1700 1 -- start start 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.961+0000 7f38aeff1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.961+0000 7f38aeff1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f38a810c4c0 con 0x7f38a8107f70 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.961+0000 7f38acd8d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.961+0000 7f38acd8d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a819be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44090/0 (socket says 172.21.15.49:44090) 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.961+0000 7f38acd8d700 1 -- 172.21.15.49:0/2845735906 learned_addr learned my addr 172.21.15.49:0/2845735906 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.961+0000 7f38acd8d700 1 -- 172.21.15.49:0/2845735906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a819c380 con 0x7f38a8107f70 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.962+0000 7f38acd8d700 1 --2- 172.21.15.49:0/2845735906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a819be40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f389c011670 tx=0x7f389c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.962+0000 7f38a5ffb700 1 -- 172.21.15.49:0/2845735906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f389c0039c0 con 0x7f38a8107f70 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.962+0000 7f38a5ffb700 1 -- 172.21.15.49:0/2845735906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f389c011e50 con 0x7f38a8107f70 2024-03-26T17:52:20.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.962+0000 7f38aeff1700 1 -- 172.21.15.49:0/2845735906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a819c610 con 0x7f38a8107f70 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.962+0000 7f38a5ffb700 1 -- 172.21.15.49:0/2845735906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f389c01a5c0 con 0x7f38a8107f70 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.962+0000 7f38aeff1700 1 -- 172.21.15.49:0/2845735906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a819f300 con 0x7f38a8107f70 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.963+0000 7f38a5ffb700 1 -- 172.21.15.49:0/2845735906 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f389c024430 con 0x7f38a8107f70 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.963+0000 7f38aeff1700 1 -- 172.21.15.49:0/2845735906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a80694b0 con 0x7f38a8107f70 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.963+0000 7f38a5ffb700 1 --2- 172.21.15.49:0/2845735906 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f3898039180 0x7f389803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.963+0000 7f38a5ffb700 1 -- 172.21.15.49:0/2845735906 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f389c04d590 con 0x7f38a8107f70 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.963+0000 7f38a7fff700 1 --2- 172.21.15.49:0/2845735906 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f3898039180 0x7f389803b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.964+0000 7f38a7fff700 1 --2- 172.21.15.49:0/2845735906 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f3898039180 0x7f389803b640 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3894006fd0 tx=0x7f3894006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:19.968+0000 7f38a5ffb700 1 -- 172.21.15.49:0/2845735906 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f389c00f2e0 con 0x7f38a8107f70 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.173+0000 7f38aeff1700 1 -- 172.21.15.49:0/2845735906 --> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f38a81046c0 con 0x7f3898039180 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.232+0000 7f38a5ffb700 1 -- 172.21.15.49:0/2845735906 <== mgr.14120 v2:172.21.15.49:6800/4249557516 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f38a81046c0 con 0x7f3898039180 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.234+0000 7f38aeff1700 1 -- 172.21.15.49:0/2845735906 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f3898039180 msgr2=0x7f389803b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:20.775 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.234+0000 7f38aeff1700 1 --2- 172.21.15.49:0/2845735906 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f3898039180 0x7f389803b640 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3894006fd0 tx=0x7f3894006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.234+0000 7f38aeff1700 1 -- 172.21.15.49:0/2845735906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 msgr2=0x7f38a819be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.234+0000 7f38aeff1700 1 --2- 172.21.15.49:0/2845735906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a819be40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f389c011670 tx=0x7f389c011b60 comp rx=0 tx=0).stop 2024-03-26T17:52:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.234+0000 7f38aeff1700 1 -- 172.21.15.49:0/2845735906 shutdown_connections 2024-03-26T17:52:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.234+0000 7f38aeff1700 1 --2- 172.21.15.49:0/2845735906 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f3898039180 0x7f389803b640 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.234+0000 7f38aeff1700 1 --2- 172.21.15.49:0/2845735906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38a8107f70 0x7f38a819be40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.234+0000 7f38aeff1700 1 -- 172.21.15.49:0/2845735906 >> 172.21.15.49:0/2845735906 conn(0x7f38a8103510 msgr2=0x7f38a8103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.234+0000 7f38aeff1700 1 -- 172.21.15.49:0/2845735906 shutdown_connections 2024-03-26T17:52:20.776 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:20.234+0000 7f38aeff1700 1 -- 172.21.15.49:0/2845735906 wait complete. 2024-03-26T17:52:21.358 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:21 smithi049 ceph-mon[32331]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:52:21.358 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:21 smithi049 ceph-mon[32331]: Generating ssh key... 2024-03-26T17:52:21.358 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:21 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:21.358 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:21 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:22.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSkR9ilOPddk77g+new7Gm/JuT9LZI1Ur9jDrIcW2YhHbPfa3ZePyA38nZVGG3UxF8N7H07V1w9ckXPb3bI2Tx7vQYpTb4WemiVl+MNsri7kFfMbyErMR/xXuszw6rQ2DURMNdsg7wogRivb9BkrRl02H8ehtyCuR8G0XZNRzF60oav1d1B7aKg10Wy2Arj/PKzNywbYeXBKUUkfmvzfuTD9/BkU/6KATE2UKiyyb78W/R++6mhtnz2ywOXEYoX8v7R89q1vidtvSRCOAAeanwO0x56CWgJQ5wftmznrj0ScsE2Sa1B/NAiGSa3P9xtliqTJ4rs8NJRsDqtDTvEmBsuQKIh/aEywEnxvUYBUMz16jdx551LBgp/bSJSun9aCHCBd3W9JFeD0PiWHTfZ85kJS60vtKvCSBHOBnRiHo2gl1jh6MiUP6Jw9/gMgbYUC94lf6YUGWNSBUR11AEYes0YniFQK4xjAVeVf/ArefPw8glBgt26gGvQOKE4zBUN6c= ceph-5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:52:22.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.148+0000 7f1fec66f700 1 Processor -- start 2024-03-26T17:52:22.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.148+0000 7f1fec66f700 1 -- start start 2024-03-26T17:52:22.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.149+0000 7f1fec66f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe410ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:22.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.149+0000 7f1fec66f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe407c620 con 0x7f1fe41089b0 2024-03-26T17:52:22.022 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.149+0000 7f1fea40b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe410ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.149+0000 7f1fea40b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe410ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44096/0 (socket says 172.21.15.49:44096) 2024-03-26T17:52:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.149+0000 7f1fea40b700 1 -- 172.21.15.49:0/1006781921 learned_addr learned my addr 172.21.15.49:0/1006781921 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.149+0000 7f1fea40b700 1 -- 172.21.15.49:0/1006781921 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe410b2e0 con 0x7f1fe41089b0 2024-03-26T17:52:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.149+0000 7f1fea40b700 1 --2- 172.21.15.49:0/1006781921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe410ada0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098a0 tx=0x7f1fd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd24a8e8ef0be426 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.150+0000 7f1fe9409700 1 -- 172.21.15.49:0/1006781921 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd8004160 con 0x7f1fe41089b0 2024-03-26T17:52:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.150+0000 7f1fe9409700 1 -- 172.21.15.49:0/1006781921 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1fd8004300 con 0x7f1fe41089b0 2024-03-26T17:52:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.150+0000 7f1fe9409700 1 -- 172.21.15.49:0/1006781921 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd8011b50 con 0x7f1fe41089b0 2024-03-26T17:52:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.150+0000 7f1fec66f700 1 -- 172.21.15.49:0/1006781921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 msgr2=0x7f1fe410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.150+0000 7f1fec66f700 1 --2- 172.21.15.49:0/1006781921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe410ada0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098a0 tx=0x7f1fd8009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.150+0000 7f1fec66f700 1 -- 172.21.15.49:0/1006781921 shutdown_connections 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.150+0000 7f1fec66f700 1 --2- 172.21.15.49:0/1006781921 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe410ada0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.150+0000 7f1fec66f700 1 -- 172.21.15.49:0/1006781921 >> 172.21.15.49:0/1006781921 conn(0x7f1fe4102400 msgr2=0x7f1fe4104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.150+0000 7f1fec66f700 1 -- 172.21.15.49:0/1006781921 shutdown_connections 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.150+0000 7f1fec66f700 1 -- 172.21.15.49:0/1006781921 wait complete. 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.151+0000 7f1fec66f700 1 Processor -- start 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.151+0000 7f1fec66f700 1 -- start start 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.151+0000 7f1fec66f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe4108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.151+0000 7f1fec66f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe410c300 con 0x7f1fe41089b0 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.151+0000 7f1fea40b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe4108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.151+0000 7f1fea40b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe4108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44098/0 (socket says 172.21.15.49:44098) 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.151+0000 7f1fea40b700 1 -- 172.21.15.49:0/2092659045 learned_addr learned my addr 172.21.15.49:0/2092659045 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.152+0000 7f1fea40b700 1 -- 172.21.15.49:0/2092659045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe4108770 con 0x7f1fe41089b0 2024-03-26T17:52:22.024 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.152+0000 7f1fea40b700 1 --2- 172.21.15.49:0/2092659045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe4108230 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009fb0 tx=0x7f1fd8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.152+0000 7f1fd77fe700 1 -- 172.21.15.49:0/2092659045 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd80038b0 con 0x7f1fe41089b0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.152+0000 7f1fd77fe700 1 -- 172.21.15.49:0/2092659045 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1fd8011e50 con 0x7f1fe41089b0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.152+0000 7f1fec66f700 1 -- 172.21.15.49:0/2092659045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe4106960 con 0x7f1fe41089b0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.152+0000 7f1fd77fe700 1 -- 172.21.15.49:0/2092659045 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd8019420 con 0x7f1fe41089b0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.152+0000 7f1fec66f700 1 -- 172.21.15.49:0/2092659045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe4106ea0 con 0x7f1fe41089b0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.153+0000 7f1fd77fe700 1 -- 172.21.15.49:0/2092659045 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f1fd80196d0 con 0x7f1fe41089b0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.153+0000 7f1fec66f700 1 -- 172.21.15.49:0/2092659045 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe40694b0 con 0x7f1fe41089b0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.153+0000 7f1fd77fe700 1 --2- 172.21.15.49:0/2092659045 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1fd0039180 0x7f1fd003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.153+0000 7f1fd77fe700 1 -- 172.21.15.49:0/2092659045 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1fd804d570 con 0x7f1fe41089b0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.153+0000 7f1fe9c0a700 1 --2- 172.21.15.49:0/2092659045 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1fd0039180 0x7f1fd003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.153+0000 7f1fe9c0a700 1 --2- 172.21.15.49:0/2092659045 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1fd0039180 0x7f1fd003b640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0006fd0 tx=0x7f1fe0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:22.025 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.158+0000 7f1fd77fe700 1 -- 172.21.15.49:0/2092659045 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1fd8003ac0 con 0x7f1fe41089b0 2024-03-26T17:52:22.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.366+0000 7f1fec66f700 1 -- 172.21.15.49:0/2092659045 --> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f1fe41036c0 con 0x7f1fd0039180 2024-03-26T17:52:22.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.367+0000 7f1fd77fe700 1 -- 172.21.15.49:0/2092659045 <== mgr.14120 v2:172.21.15.49:6800/4249557516 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f1fe41036c0 con 0x7f1fd0039180 2024-03-26T17:52:22.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.369+0000 7f1fec66f700 1 -- 172.21.15.49:0/2092659045 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1fd0039180 msgr2=0x7f1fd003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:22.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.369+0000 7f1fec66f700 1 --2- 172.21.15.49:0/2092659045 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1fd0039180 0x7f1fd003b640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0006fd0 tx=0x7f1fe0006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:22.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.369+0000 7f1fec66f700 1 -- 172.21.15.49:0/2092659045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 msgr2=0x7f1fe4108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:22.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.369+0000 7f1fec66f700 1 --2- 172.21.15.49:0/2092659045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe4108230 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009fb0 tx=0x7f1fd8011bf0 comp rx=0 tx=0).stop 2024-03-26T17:52:22.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.369+0000 7f1fec66f700 1 -- 172.21.15.49:0/2092659045 shutdown_connections 2024-03-26T17:52:22.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.369+0000 7f1fec66f700 1 --2- 172.21.15.49:0/2092659045 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1fd0039180 0x7f1fd003b640 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:22.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.369+0000 7f1fec66f700 1 --2- 172.21.15.49:0/2092659045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1fe41089b0 0x7f1fe4108230 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:22.026 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.369+0000 7f1fec66f700 1 -- 172.21.15.49:0/2092659045 >> 172.21.15.49:0/2092659045 conn(0x7f1fe4102400 msgr2=0x7f1fe4102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:22.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.370+0000 7f1fec66f700 1 -- 172.21.15.49:0/2092659045 shutdown_connections 2024-03-26T17:52:22.027 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:21.370+0000 7f1fec66f700 1 -- 172.21.15.49:0/2092659045 wait complete. 2024-03-26T17:52:22.027 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-26T17:52:22.027 INFO:teuthology.orchestra.run.smithi049.stdout:Adding key to root@localhost authorized_keys... 2024-03-26T17:52:22.027 INFO:teuthology.orchestra.run.smithi049.stdout:Adding host smithi049... 2024-03-26T17:52:22.299 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:22 smithi049 ceph-mon[32331]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:52:23.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:23 smithi049 ceph-mon[32331]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:52:23.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:23 smithi049 ceph-mon[32331]: Deploying cephadm binary to smithi049 2024-03-26T17:52:27.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Added host 'smithi049' with addr '172.21.15.49' 2024-03-26T17:52:27.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.419+0000 7ff074b19700 1 Processor -- start 2024-03-26T17:52:27.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.419+0000 7ff074b19700 1 -- start start 2024-03-26T17:52:27.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.419+0000 7ff074b19700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff0701083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:27.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.419+0000 7ff074b19700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701088e0 con 0x7ff070107f80 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.419+0000 7ff06e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff0701083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.419+0000 7ff06e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff0701083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44110/0 (socket says 172.21.15.49:44110) 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.419+0000 7ff06e59c700 1 -- 172.21.15.49:0/1208583394 learned_addr learned my addr 172.21.15.49:0/1208583394 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.419+0000 7ff06e59c700 1 -- 172.21.15.49:0/1208583394 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff070109090 con 0x7ff070107f80 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.420+0000 7ff06e59c700 1 --2- 172.21.15.49:0/1208583394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff0701083a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff0600098a0 tx=0x7ff060009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45531f0192e608a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.420+0000 7ff06d59a700 1 -- 172.21.15.49:0/1208583394 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff060004160 con 0x7ff070107f80 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.420+0000 7ff06d59a700 1 -- 172.21.15.49:0/1208583394 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff060004300 con 0x7ff070107f80 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.420+0000 7ff06d59a700 1 -- 172.21.15.49:0/1208583394 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff060011b50 con 0x7ff070107f80 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.420+0000 7ff074b19700 1 -- 172.21.15.49:0/1208583394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 msgr2=0x7ff0701083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.420+0000 7ff074b19700 1 --2- 172.21.15.49:0/1208583394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff0701083a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff0600098a0 tx=0x7ff060009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:27.705 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.421+0000 7ff074b19700 1 -- 172.21.15.49:0/1208583394 shutdown_connections 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.421+0000 7ff074b19700 1 --2- 172.21.15.49:0/1208583394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff0701083a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.421+0000 7ff074b19700 1 -- 172.21.15.49:0/1208583394 >> 172.21.15.49:0/1208583394 conn(0x7ff070103540 msgr2=0x7ff070105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.421+0000 7ff074b19700 1 -- 172.21.15.49:0/1208583394 shutdown_connections 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.421+0000 7ff074b19700 1 -- 172.21.15.49:0/1208583394 wait complete. 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.421+0000 7ff074b19700 1 Processor -- start 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.421+0000 7ff074b19700 1 -- start start 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff074b19700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff07019be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff074b19700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff07010c4d0 con 0x7ff070107f80 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff06e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff07019be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff06e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff07019be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44114/0 (socket says 172.21.15.49:44114) 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff06e59c700 1 -- 172.21.15.49:0/3341744026 learned_addr learned my addr 172.21.15.49:0/3341744026 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:27.706 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff06e59c700 1 -- 172.21.15.49:0/3341744026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07019c3d0 con 0x7ff070107f80 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff06e59c700 1 --2- 172.21.15.49:0/3341744026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff07019be90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff060011b60 tx=0x7ff060011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff05f7fe700 1 -- 172.21.15.49:0/3341744026 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff060003b20 con 0x7ff070107f80 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff074b19700 1 -- 172.21.15.49:0/3341744026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07019c660 con 0x7ff070107f80 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff05f7fe700 1 -- 172.21.15.49:0/3341744026 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff060003cc0 con 0x7ff070107f80 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff05f7fe700 1 -- 172.21.15.49:0/3341744026 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff060024840 con 0x7ff070107f80 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.422+0000 7ff074b19700 1 -- 172.21.15.49:0/3341744026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07019f350 con 0x7ff070107f80 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.423+0000 7ff05f7fe700 1 -- 172.21.15.49:0/3341744026 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7ff060022020 con 0x7ff070107f80 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.423+0000 7ff05f7fe700 1 --2- 172.21.15.49:0/3341744026 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7ff058039180 0x7ff05803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.423+0000 7ff074b19700 1 -- 172.21.15.49:0/3341744026 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0700639e0 con 0x7ff070107f80 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.424+0000 7ff05f7fe700 1 -- 172.21.15.49:0/3341744026 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff060011e00 con 0x7ff070107f80 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.424+0000 7ff06dd9b700 1 --2- 172.21.15.49:0/3341744026 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7ff058039180 0x7ff05803b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.424+0000 7ff06dd9b700 1 --2- 172.21.15.49:0/3341744026 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7ff058039180 0x7ff05803b640 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff064006fd0 tx=0x7ff064006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:27.707 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.429+0000 7ff05f7fe700 1 -- 172.21.15.49:0/3341744026 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff060023370 con 0x7ff070107f80 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:22.636+0000 7ff074b19700 1 -- 172.21.15.49:0/3341744026 --> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}) -- 0x7ff0701046d0 con 0x7ff058039180 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.107+0000 7ff05f7fe700 1 -- 172.21.15.49:0/3341744026 <== mgr.14120 v2:172.21.15.49:6800/4249557516 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff0701046d0 con 0x7ff058039180 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.110+0000 7ff074b19700 1 -- 172.21.15.49:0/3341744026 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7ff058039180 msgr2=0x7ff05803b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.110+0000 7ff074b19700 1 --2- 172.21.15.49:0/3341744026 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7ff058039180 0x7ff05803b640 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff064006fd0 tx=0x7ff064006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.110+0000 7ff074b19700 1 -- 172.21.15.49:0/3341744026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 msgr2=0x7ff07019be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.110+0000 7ff074b19700 1 --2- 172.21.15.49:0/3341744026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff07019be90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff060011b60 tx=0x7ff060011c40 comp rx=0 tx=0).stop 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.110+0000 7ff074b19700 1 -- 172.21.15.49:0/3341744026 shutdown_connections 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.110+0000 7ff074b19700 1 --2- 172.21.15.49:0/3341744026 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7ff058039180 0x7ff05803b640 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.110+0000 7ff074b19700 1 --2- 172.21.15.49:0/3341744026 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff070107f80 0x7ff07019be90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.110+0000 7ff074b19700 1 -- 172.21.15.49:0/3341744026 >> 172.21.15.49:0/3341744026 conn(0x7ff070103540 msgr2=0x7ff070103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.110+0000 7ff074b19700 1 -- 172.21.15.49:0/3341744026 shutdown_connections 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:27.110+0000 7ff074b19700 1 -- 172.21.15.49:0/3341744026 wait complete. 2024-03-26T17:52:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mon service... 2024-03-26T17:52:28.261 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:28 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:28.261 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:28 smithi049 ceph-mon[32331]: Added host smithi049 2024-03-26T17:52:28.261 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:28 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:29.180 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-26T17:52:29.180 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.114+0000 7fd42a737700 1 Processor -- start 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.115+0000 7fd42a737700 1 -- start start 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.115+0000 7fd42a737700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42410ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.115+0000 7fd42a737700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd42407c630 con 0x7fd424108990 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.115+0000 7fd423fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42410ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.115+0000 7fd423fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42410ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44116/0 (socket says 172.21.15.49:44116) 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.115+0000 7fd423fff700 1 -- 172.21.15.49:0/21622464 learned_addr learned my addr 172.21.15.49:0/21622464 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.115+0000 7fd423fff700 1 -- 172.21.15.49:0/21622464 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42410b2c0 con 0x7fd424108990 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.115+0000 7fd423fff700 1 --2- 172.21.15.49:0/21622464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42410ad80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd4140098a0 tx=0x7fd414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1ea20fa3cd90f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.116+0000 7fd422ffd700 1 -- 172.21.15.49:0/21622464 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd414004160 con 0x7fd424108990 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.116+0000 7fd422ffd700 1 -- 172.21.15.49:0/21622464 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd414004300 con 0x7fd424108990 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.116+0000 7fd422ffd700 1 -- 172.21.15.49:0/21622464 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd414011b50 con 0x7fd424108990 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.116+0000 7fd42a737700 1 -- 172.21.15.49:0/21622464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 msgr2=0x7fd42410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:29.181 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.116+0000 7fd42a737700 1 --2- 172.21.15.49:0/21622464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42410ad80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd4140098a0 tx=0x7fd414009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.116+0000 7fd42a737700 1 -- 172.21.15.49:0/21622464 shutdown_connections 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.116+0000 7fd42a737700 1 --2- 172.21.15.49:0/21622464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42410ad80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.116+0000 7fd42a737700 1 -- 172.21.15.49:0/21622464 >> 172.21.15.49:0/21622464 conn(0x7fd424102400 msgr2=0x7fd424104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.116+0000 7fd42a737700 1 -- 172.21.15.49:0/21622464 shutdown_connections 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.116+0000 7fd42a737700 1 -- 172.21.15.49:0/21622464 wait complete. 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.117+0000 7fd42a737700 1 Processor -- start 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.117+0000 7fd42a737700 1 -- start start 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.117+0000 7fd42a737700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.117+0000 7fd42a737700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd42410c2e0 con 0x7fd424108990 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.118+0000 7fd423fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.118+0000 7fd423fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42419be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:44118/0 (socket says 172.21.15.49:44118) 2024-03-26T17:52:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.118+0000 7fd423fff700 1 -- 172.21.15.49:0/738172972 learned_addr learned my addr 172.21.15.49:0/738172972 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.118+0000 7fd423fff700 1 -- 172.21.15.49:0/738172972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42419c390 con 0x7fd424108990 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.118+0000 7fd423fff700 1 --2- 172.21.15.49:0/738172972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42419be50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd414011670 tx=0x7fd414011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.118+0000 7fd4217fa700 1 -- 172.21.15.49:0/738172972 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4140039c0 con 0x7fd424108990 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.118+0000 7fd4217fa700 1 -- 172.21.15.49:0/738172972 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd414011e50 con 0x7fd424108990 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.118+0000 7fd42a737700 1 -- 172.21.15.49:0/738172972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42419c620 con 0x7fd424108990 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.118+0000 7fd4217fa700 1 -- 172.21.15.49:0/738172972 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd41401a5c0 con 0x7fd424108990 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.118+0000 7fd42a737700 1 -- 172.21.15.49:0/738172972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42419f310 con 0x7fd424108990 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.119+0000 7fd4217fa700 1 -- 172.21.15.49:0/738172972 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7fd41401a870 con 0x7fd424108990 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.119+0000 7fd42a737700 1 -- 172.21.15.49:0/738172972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4240639e0 con 0x7fd424108990 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.119+0000 7fd4217fa700 1 --2- 172.21.15.49:0/738172972 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd40c039180 0x7fd40c03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.119+0000 7fd4217fa700 1 -- 172.21.15.49:0/738172972 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd41404d680 con 0x7fd424108990 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.119+0000 7fd4237fe700 1 --2- 172.21.15.49:0/738172972 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd40c039180 0x7fd40c03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.120+0000 7fd4237fe700 1 --2- 172.21.15.49:0/738172972 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd40c039180 0x7fd40c03b640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd418006fd0 tx=0x7fd418006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:29.183 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.125+0000 7fd4217fa700 1 -- 172.21.15.49:0/738172972 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd414019c70 con 0x7fd424108990 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.347+0000 7fd42a737700 1 -- 172.21.15.49:0/738172972 --> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd424103620 con 0x7fd40c039180 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.352+0000 7fd4217fa700 1 -- 172.21.15.49:0/738172972 <== mgr.14120 v2:172.21.15.49:6800/4249557516 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd424103620 con 0x7fd40c039180 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.354+0000 7fd42a737700 1 -- 172.21.15.49:0/738172972 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd40c039180 msgr2=0x7fd40c03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.354+0000 7fd42a737700 1 --2- 172.21.15.49:0/738172972 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd40c039180 0x7fd40c03b640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd418006fd0 tx=0x7fd418006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.354+0000 7fd42a737700 1 -- 172.21.15.49:0/738172972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 msgr2=0x7fd42419be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.354+0000 7fd42a737700 1 --2- 172.21.15.49:0/738172972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42419be50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd414011670 tx=0x7fd414011b60 comp rx=0 tx=0).stop 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.354+0000 7fd42a737700 1 -- 172.21.15.49:0/738172972 shutdown_connections 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.354+0000 7fd42a737700 1 --2- 172.21.15.49:0/738172972 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd40c039180 0x7fd40c03b640 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.354+0000 7fd42a737700 1 --2- 172.21.15.49:0/738172972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd424108990 0x7fd42419be50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.354+0000 7fd42a737700 1 -- 172.21.15.49:0/738172972 >> 172.21.15.49:0/738172972 conn(0x7fd424102400 msgr2=0x7fd424102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.354+0000 7fd42a737700 1 -- 172.21.15.49:0/738172972 shutdown_connections 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:28.354+0000 7fd42a737700 1 -- 172.21.15.49:0/738172972 wait complete. 2024-03-26T17:52:29.184 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mgr service... 2024-03-26T17:52:29.450 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:29 smithi049 ceph-mon[32331]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:52:29.450 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:29 smithi049 ceph-mon[32331]: Saving service mon spec with placement count:5 2024-03-26T17:52:29.450 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:29 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:30.457 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-26T17:52:30.457 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.565+0000 7f45870dc700 1 Processor -- start 2024-03-26T17:52:30.457 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.565+0000 7f45870dc700 1 -- start start 2024-03-26T17:52:30.457 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.565+0000 7f45870dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458007cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:30.457 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.565+0000 7f45870dc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f458007d000 con 0x7f458007e5c0 2024-03-26T17:52:30.457 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.565+0000 7f4584e78700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458007cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:30.457 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.565+0000 7f4584e78700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458007cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42306/0 (socket says 172.21.15.49:42306) 2024-03-26T17:52:30.457 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.565+0000 7f4584e78700 1 -- 172.21.15.49:0/2939416809 learned_addr learned my addr 172.21.15.49:0/2939416809 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:30.457 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.565+0000 7f4584e78700 1 -- 172.21.15.49:0/2939416809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458007d180 con 0x7f458007e5c0 2024-03-26T17:52:30.457 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.566+0000 7f4584e78700 1 --2- 172.21.15.49:0/2939416809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458007cac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45740098a0 tx=0x7f4574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcac256ee0ee11ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.566+0000 7f457f7fe700 1 -- 172.21.15.49:0/2939416809 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4574004160 con 0x7f458007e5c0 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.566+0000 7f457f7fe700 1 -- 172.21.15.49:0/2939416809 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4574004300 con 0x7f458007e5c0 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.566+0000 7f457f7fe700 1 -- 172.21.15.49:0/2939416809 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4574011b50 con 0x7f458007e5c0 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.566+0000 7f45870dc700 1 -- 172.21.15.49:0/2939416809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 msgr2=0x7f458007cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.566+0000 7f45870dc700 1 --2- 172.21.15.49:0/2939416809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458007cac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45740098a0 tx=0x7f4574009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.566+0000 7f45870dc700 1 -- 172.21.15.49:0/2939416809 shutdown_connections 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.566+0000 7f45870dc700 1 --2- 172.21.15.49:0/2939416809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458007cac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.566+0000 7f45870dc700 1 -- 172.21.15.49:0/2939416809 >> 172.21.15.49:0/2939416809 conn(0x7f45801035b0 msgr2=0x7f45801059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.567+0000 7f45870dc700 1 -- 172.21.15.49:0/2939416809 shutdown_connections 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.567+0000 7f45870dc700 1 -- 172.21.15.49:0/2939416809 wait complete. 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.567+0000 7f45870dc700 1 Processor -- start 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.567+0000 7f45870dc700 1 -- start start 2024-03-26T17:52:30.458 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.567+0000 7f45870dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.567+0000 7f45870dc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f458010e560 con 0x7f458007e5c0 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.567+0000 7f4584e78700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.567+0000 7f4584e78700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42308/0 (socket says 172.21.15.49:42308) 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.567+0000 7f4584e78700 1 -- 172.21.15.49:0/2115483710 learned_addr learned my addr 172.21.15.49:0/2115483710 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.568+0000 7f4584e78700 1 -- 172.21.15.49:0/2115483710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458019e5d0 con 0x7f458007e5c0 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.568+0000 7f4584e78700 1 --2- 172.21.15.49:0/2115483710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458019e090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4574011b60 tx=0x7f4574011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.568+0000 7f457dffb700 1 -- 172.21.15.49:0/2115483710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4574003b20 con 0x7f458007e5c0 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.568+0000 7f457dffb700 1 -- 172.21.15.49:0/2115483710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4574003cc0 con 0x7f458007e5c0 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.568+0000 7f45870dc700 1 -- 172.21.15.49:0/2115483710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458019e860 con 0x7f458007e5c0 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.568+0000 7f457dffb700 1 -- 172.21.15.49:0/2115483710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4574024840 con 0x7f458007e5c0 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.568+0000 7f45870dc700 1 -- 172.21.15.49:0/2115483710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45801a1550 con 0x7f458007e5c0 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.569+0000 7f457dffb700 1 -- 172.21.15.49:0/2115483710 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f4574022020 con 0x7f458007e5c0 2024-03-26T17:52:30.459 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.569+0000 7f45870dc700 1 -- 172.21.15.49:0/2115483710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45800639e0 con 0x7f458007e5c0 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.569+0000 7f457dffb700 1 --2- 172.21.15.49:0/2115483710 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f45700391d0 0x7f457003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.569+0000 7f457dffb700 1 -- 172.21.15.49:0/2115483710 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4574011e00 con 0x7f458007e5c0 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.569+0000 7f457ffff700 1 --2- 172.21.15.49:0/2115483710 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f45700391d0 0x7f457003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.570+0000 7f457ffff700 1 --2- 172.21.15.49:0/2115483710 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f45700391d0 0x7f457003b690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f456c006fd0 tx=0x7f456c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.574+0000 7f457dffb700 1 -- 172.21.15.49:0/2115483710 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f457400b3e0 con 0x7f458007e5c0 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.799+0000 7f45870dc700 1 -- 172.21.15.49:0/2115483710 --> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4580104830 con 0x7f45700391d0 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.803+0000 7f457dffb700 1 -- 172.21.15.49:0/2115483710 <== mgr.14120 v2:172.21.15.49:6800/4249557516 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4580104830 con 0x7f45700391d0 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.805+0000 7f45870dc700 1 -- 172.21.15.49:0/2115483710 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f45700391d0 msgr2=0x7f457003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.805+0000 7f45870dc700 1 --2- 172.21.15.49:0/2115483710 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f45700391d0 0x7f457003b690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f456c006fd0 tx=0x7f456c006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.805+0000 7f45870dc700 1 -- 172.21.15.49:0/2115483710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 msgr2=0x7f458019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.805+0000 7f45870dc700 1 --2- 172.21.15.49:0/2115483710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458019e090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4574011b60 tx=0x7f4574011c40 comp rx=0 tx=0).stop 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.805+0000 7f45870dc700 1 -- 172.21.15.49:0/2115483710 shutdown_connections 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.805+0000 7f45870dc700 1 --2- 172.21.15.49:0/2115483710 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f45700391d0 0x7f457003b690 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:30.460 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.805+0000 7f45870dc700 1 --2- 172.21.15.49:0/2115483710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f458007e5c0 0x7f458019e090 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:30.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.805+0000 7f45870dc700 1 -- 172.21.15.49:0/2115483710 >> 172.21.15.49:0/2115483710 conn(0x7f45801035b0 msgr2=0x7f4580104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:30.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.806+0000 7f45870dc700 1 -- 172.21.15.49:0/2115483710 shutdown_connections 2024-03-26T17:52:30.461 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:29.806+0000 7f45870dc700 1 -- 172.21.15.49:0/2115483710 wait complete. 2024-03-26T17:52:31.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:30 smithi049 ceph-mon[32331]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:52:31.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:30 smithi049 ceph-mon[32331]: Saving service mgr spec with placement count:2 2024-03-26T17:52:31.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:30 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:31.814 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.901+0000 7f152e28e700 1 Processor -- start 2024-03-26T17:52:31.814 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.901+0000 7f152e28e700 1 -- start start 2024-03-26T17:52:31.814 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.901+0000 7f152e28e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f1528107250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:31.814 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.901+0000 7f152e28e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1528107790 con 0x7f1528106e30 2024-03-26T17:52:31.814 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.901+0000 7f1527fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f1528107250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.901+0000 7f1527fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f1528107250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42324/0 (socket says 172.21.15.49:42324) 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.901+0000 7f1527fff700 1 -- 172.21.15.49:0/1100280655 learned_addr learned my addr 172.21.15.49:0/1100280655 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.902+0000 7f1527fff700 1 -- 172.21.15.49:0/1100280655 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1528107f90 con 0x7f1528106e30 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.902+0000 7f1527fff700 1 --2- 172.21.15.49:0/1100280655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f1528107250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f15180098a0 tx=0x7f1518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac64aa3b585e9b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.902+0000 7f1526ffd700 1 -- 172.21.15.49:0/1100280655 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1518004160 con 0x7f1528106e30 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.902+0000 7f1526ffd700 1 -- 172.21.15.49:0/1100280655 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1518004300 con 0x7f1528106e30 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.902+0000 7f1526ffd700 1 -- 172.21.15.49:0/1100280655 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1518011b50 con 0x7f1528106e30 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.903+0000 7f152e28e700 1 -- 172.21.15.49:0/1100280655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 msgr2=0x7f1528107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.903+0000 7f152e28e700 1 --2- 172.21.15.49:0/1100280655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f1528107250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f15180098a0 tx=0x7f1518009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.903+0000 7f152e28e700 1 -- 172.21.15.49:0/1100280655 shutdown_connections 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.903+0000 7f152e28e700 1 --2- 172.21.15.49:0/1100280655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f1528107250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:31.815 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.903+0000 7f152e28e700 1 -- 172.21.15.49:0/1100280655 >> 172.21.15.49:0/1100280655 conn(0x7f1528102450 msgr2=0x7f1528104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.903+0000 7f152e28e700 1 -- 172.21.15.49:0/1100280655 shutdown_connections 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.903+0000 7f152e28e700 1 -- 172.21.15.49:0/1100280655 wait complete. 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.903+0000 7f152e28e700 1 Processor -- start 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.903+0000 7f152e28e700 1 -- start start 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.904+0000 7f152e28e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f152807c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.904+0000 7f152e28e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f152810b3d0 con 0x7f1528106e30 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.904+0000 7f1527fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f152807c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.904+0000 7f1527fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f152807c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42334/0 (socket says 172.21.15.49:42334) 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.904+0000 7f1527fff700 1 -- 172.21.15.49:0/1847760909 learned_addr learned my addr 172.21.15.49:0/1847760909 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.904+0000 7f1527fff700 1 -- 172.21.15.49:0/1847760909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152819ec00 con 0x7f1528106e30 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.904+0000 7f1527fff700 1 --2- 172.21.15.49:0/1847760909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f152807c210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1518011b60 tx=0x7f1518011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.905+0000 7f15257fa700 1 -- 172.21.15.49:0/1847760909 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1518003970 con 0x7f1528106e30 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.905+0000 7f15257fa700 1 -- 172.21.15.49:0/1847760909 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1518003b10 con 0x7f1528106e30 2024-03-26T17:52:31.816 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.905+0000 7f152e28e700 1 -- 172.21.15.49:0/1847760909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152819edd0 con 0x7f1528106e30 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.905+0000 7f15257fa700 1 -- 172.21.15.49:0/1847760909 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1518024610 con 0x7f1528106e30 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.905+0000 7f152e28e700 1 -- 172.21.15.49:0/1847760909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152819f170 con 0x7f1528106e30 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.905+0000 7f15257fa700 1 -- 172.21.15.49:0/1847760909 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f1518024910 con 0x7f1528106e30 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.906+0000 7f152e28e700 1 -- 172.21.15.49:0/1847760909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15280694b0 con 0x7f1528106e30 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.906+0000 7f15257fa700 1 --2- 172.21.15.49:0/1847760909 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1510039180 0x7f151003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.906+0000 7f15257fa700 1 -- 172.21.15.49:0/1847760909 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1518011e00 con 0x7f1528106e30 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.906+0000 7f15277fe700 1 --2- 172.21.15.49:0/1847760909 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1510039180 0x7f151003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.906+0000 7f15277fe700 1 --2- 172.21.15.49:0/1847760909 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1510039180 0x7f151003b640 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f151c006fd0 tx=0x7f151c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:30.911+0000 7f15257fa700 1 -- 172.21.15.49:0/1847760909 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f151800ba00 con 0x7f1528106e30 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.120+0000 7f152e28e700 1 -- 172.21.15.49:0/1847760909 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f152819f3f0 con 0x7f1528106e30 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.122+0000 7f15257fa700 1 -- 172.21.15.49:0/1847760909 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f151800bc60 con 0x7f1528106e30 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.125+0000 7f152e28e700 1 -- 172.21.15.49:0/1847760909 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1510039180 msgr2=0x7f151003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:31.817 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.125+0000 7f152e28e700 1 --2- 172.21.15.49:0/1847760909 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1510039180 0x7f151003b640 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f151c006fd0 tx=0x7f151c006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:31.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.125+0000 7f152e28e700 1 -- 172.21.15.49:0/1847760909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 msgr2=0x7f152807c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:31.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.125+0000 7f152e28e700 1 --2- 172.21.15.49:0/1847760909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f152807c210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1518011b60 tx=0x7f1518011c40 comp rx=0 tx=0).stop 2024-03-26T17:52:31.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.125+0000 7f152e28e700 1 -- 172.21.15.49:0/1847760909 shutdown_connections 2024-03-26T17:52:31.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.125+0000 7f152e28e700 1 --2- 172.21.15.49:0/1847760909 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7f1510039180 0x7f151003b640 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:31.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.125+0000 7f152e28e700 1 --2- 172.21.15.49:0/1847760909 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1528106e30 0x7f152807c210 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:31.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.125+0000 7f152e28e700 1 -- 172.21.15.49:0/1847760909 >> 172.21.15.49:0/1847760909 conn(0x7f1528102450 msgr2=0x7f1528102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:31.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.125+0000 7f152e28e700 1 -- 172.21.15.49:0/1847760909 shutdown_connections 2024-03-26T17:52:31.818 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:31.125+0000 7f152e28e700 1 -- 172.21.15.49:0/1847760909 wait complete. 2024-03-26T17:52:32.187 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:32 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1847760909' entity='client.admin' 2024-03-26T17:52:32.187 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:32 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:32.187 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:32 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:33.724 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:33 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3001917415' entity='client.admin' 2024-03-26T17:52:33.725 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:33 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:33.732 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.299+0000 7fb719d97700 1 Processor -- start 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.299+0000 7fb719d97700 1 -- start start 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.299+0000 7fb719d97700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb714109890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.299+0000 7fb719d97700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb714109dd0 con 0x7fb714109470 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.300+0000 7fb7137fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb714109890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.300+0000 7fb7137fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb714109890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42340/0 (socket says 172.21.15.49:42340) 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.300+0000 7fb7137fe700 1 -- 172.21.15.49:0/415442025 learned_addr learned my addr 172.21.15.49:0/415442025 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.300+0000 7fb7137fe700 1 -- 172.21.15.49:0/415442025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb714107c90 con 0x7fb714109470 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.300+0000 7fb7137fe700 1 --2- 172.21.15.49:0/415442025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb714109890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb7080098a0 tx=0x7fb708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3829d71aa52bbbb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.300+0000 7fb7127fc700 1 -- 172.21.15.49:0/415442025 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb708004160 con 0x7fb714109470 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.300+0000 7fb7127fc700 1 -- 172.21.15.49:0/415442025 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb708004300 con 0x7fb714109470 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.300+0000 7fb7127fc700 1 -- 172.21.15.49:0/415442025 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb708011b20 con 0x7fb714109470 2024-03-26T17:52:33.733 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.301+0000 7fb719d97700 1 -- 172.21.15.49:0/415442025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 msgr2=0x7fb714109890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.301+0000 7fb719d97700 1 --2- 172.21.15.49:0/415442025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb714109890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb7080098a0 tx=0x7fb708009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.301+0000 7fb719d97700 1 -- 172.21.15.49:0/415442025 shutdown_connections 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.301+0000 7fb719d97700 1 --2- 172.21.15.49:0/415442025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb714109890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.301+0000 7fb719d97700 1 -- 172.21.15.49:0/415442025 >> 172.21.15.49:0/415442025 conn(0x7fb714103630 msgr2=0x7fb714105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.301+0000 7fb719d97700 1 -- 172.21.15.49:0/415442025 shutdown_connections 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.301+0000 7fb719d97700 1 -- 172.21.15.49:0/415442025 wait complete. 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.302+0000 7fb719d97700 1 Processor -- start 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.302+0000 7fb719d97700 1 -- start start 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.302+0000 7fb719d97700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb71419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.302+0000 7fb719d97700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb71410c3b0 con 0x7fb714109470 2024-03-26T17:52:33.734 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.302+0000 7fb7137fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb71419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.302+0000 7fb7137fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb71419e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42356/0 (socket says 172.21.15.49:42356) 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.302+0000 7fb7137fe700 1 -- 172.21.15.49:0/3001917415 learned_addr learned my addr 172.21.15.49:0/3001917415 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.302+0000 7fb7137fe700 1 -- 172.21.15.49:0/3001917415 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71419e5f0 con 0x7fb714109470 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.303+0000 7fb7137fe700 1 --2- 172.21.15.49:0/3001917415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb71419e0b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb7080117e0 tx=0x7fb708011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.303+0000 7fb710ff9700 1 -- 172.21.15.49:0/3001917415 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb708003c90 con 0x7fb714109470 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.303+0000 7fb710ff9700 1 -- 172.21.15.49:0/3001917415 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb70801a910 con 0x7fb714109470 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.303+0000 7fb719d97700 1 -- 172.21.15.49:0/3001917415 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71419e880 con 0x7fb714109470 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.303+0000 7fb710ff9700 1 -- 172.21.15.49:0/3001917415 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb708024a10 con 0x7fb714109470 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.303+0000 7fb719d97700 1 -- 172.21.15.49:0/3001917415 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71419ece0 con 0x7fb714109470 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.304+0000 7fb710ff9700 1 -- 172.21.15.49:0/3001917415 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7fb708022020 con 0x7fb714109470 2024-03-26T17:52:33.735 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.304+0000 7fb719d97700 1 -- 172.21.15.49:0/3001917415 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7140639e0 con 0x7fb714109470 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.304+0000 7fb710ff9700 1 --2- 172.21.15.49:0/3001917415 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fb6fc039130 0x7fb6fc03b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.304+0000 7fb710ff9700 1 -- 172.21.15.49:0/3001917415 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb70804d540 con 0x7fb714109470 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.304+0000 7fb712ffd700 1 --2- 172.21.15.49:0/3001917415 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fb6fc039130 0x7fb6fc03b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.305+0000 7fb712ffd700 1 --2- 172.21.15.49:0/3001917415 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fb6fc039130 0x7fb6fc03b5f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb704006fd0 tx=0x7fb704006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.309+0000 7fb710ff9700 1 -- 172.21.15.49:0/3001917415 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb708011e20 con 0x7fb714109470 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.520+0000 7fb719d97700 1 -- 172.21.15.49:0/3001917415 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb7141a1690 con 0x7fb714109470 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.523+0000 7fb710ff9700 1 -- 172.21.15.49:0/3001917415 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb708024dc0 con 0x7fb714109470 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.525+0000 7fb719d97700 1 -- 172.21.15.49:0/3001917415 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fb6fc039130 msgr2=0x7fb6fc03b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.525+0000 7fb719d97700 1 --2- 172.21.15.49:0/3001917415 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fb6fc039130 0x7fb6fc03b5f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb704006fd0 tx=0x7fb704006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.525+0000 7fb719d97700 1 -- 172.21.15.49:0/3001917415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 msgr2=0x7fb71419e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:33.736 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.525+0000 7fb719d97700 1 --2- 172.21.15.49:0/3001917415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb71419e0b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb7080117e0 tx=0x7fb708011ce0 comp rx=0 tx=0).stop 2024-03-26T17:52:33.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.525+0000 7fb719d97700 1 -- 172.21.15.49:0/3001917415 shutdown_connections 2024-03-26T17:52:33.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.525+0000 7fb719d97700 1 --2- 172.21.15.49:0/3001917415 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fb6fc039130 0x7fb6fc03b5f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:33.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.525+0000 7fb719d97700 1 --2- 172.21.15.49:0/3001917415 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb714109470 0x7fb71419e0b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:33.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.525+0000 7fb719d97700 1 -- 172.21.15.49:0/3001917415 >> 172.21.15.49:0/3001917415 conn(0x7fb714103630 msgr2=0x7fb71407c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:33.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.525+0000 7fb719d97700 1 -- 172.21.15.49:0/3001917415 shutdown_connections 2024-03-26T17:52:33.737 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:32.525+0000 7fb719d97700 1 -- 172.21.15.49:0/3001917415 wait complete. 2024-03-26T17:52:33.737 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling the dashboard module... 2024-03-26T17:52:35.722 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:35 smithi049 ceph-mon[32331]: from='mgr.14120 172.21.15.49:0/1426027826' entity='mgr.x' 2024-03-26T17:52:35.722 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:35 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3903328093' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-26T17:52:35.722 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:35 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: ignoring --setuser ceph since I am not root 2024-03-26T17:52:35.722 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:35 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: ignoring --setgroup ceph since I am not root 2024-03-26T17:52:36.454 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:36 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:36.060+0000 7f3574cff200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T17:52:36.454 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:36 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:36.159+0000 7f3574cff200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.211+0000 7eff9c246700 1 Processor -- start 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.211+0000 7eff9c246700 1 -- start start 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.211+0000 7eff9c246700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff94109890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.211+0000 7eff9c246700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff94109dd0 con 0x7eff94109470 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.211+0000 7eff99fe2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff94109890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.211+0000 7eff99fe2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff94109890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42366/0 (socket says 172.21.15.49:42366) 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.212+0000 7eff99fe2700 1 -- 172.21.15.49:0/125521303 learned_addr learned my addr 172.21.15.49:0/125521303 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.212+0000 7eff99fe2700 1 -- 172.21.15.49:0/125521303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff94107c90 con 0x7eff94109470 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.212+0000 7eff99fe2700 1 --2- 172.21.15.49:0/125521303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff94109890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff900098a0 tx=0x7eff90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56508b202a7b3808 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.212+0000 7eff98fe0700 1 -- 172.21.15.49:0/125521303 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff90004160 con 0x7eff94109470 2024-03-26T17:52:36.463 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.212+0000 7eff98fe0700 1 -- 172.21.15.49:0/125521303 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff90004300 con 0x7eff94109470 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.212+0000 7eff98fe0700 1 -- 172.21.15.49:0/125521303 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff90011b20 con 0x7eff94109470 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.213+0000 7eff9c246700 1 -- 172.21.15.49:0/125521303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 msgr2=0x7eff94109890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.213+0000 7eff9c246700 1 --2- 172.21.15.49:0/125521303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff94109890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff900098a0 tx=0x7eff90009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.213+0000 7eff9c246700 1 -- 172.21.15.49:0/125521303 shutdown_connections 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.213+0000 7eff9c246700 1 --2- 172.21.15.49:0/125521303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff94109890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.213+0000 7eff9c246700 1 -- 172.21.15.49:0/125521303 >> 172.21.15.49:0/125521303 conn(0x7eff94103630 msgr2=0x7eff94105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.213+0000 7eff9c246700 1 -- 172.21.15.49:0/125521303 shutdown_connections 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.213+0000 7eff9c246700 1 -- 172.21.15.49:0/125521303 wait complete. 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.214+0000 7eff9c246700 1 Processor -- start 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.214+0000 7eff9c246700 1 -- start start 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.214+0000 7eff9c246700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff9419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.214+0000 7eff9c246700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff9410c3b0 con 0x7eff94109470 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.214+0000 7eff99fe2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff9419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:36.464 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.214+0000 7eff99fe2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff9419be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42380/0 (socket says 172.21.15.49:42380) 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.214+0000 7eff99fe2700 1 -- 172.21.15.49:0/3903328093 learned_addr learned my addr 172.21.15.49:0/3903328093 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.214+0000 7eff99fe2700 1 -- 172.21.15.49:0/3903328093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9419c3c0 con 0x7eff94109470 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.215+0000 7eff99fe2700 1 --2- 172.21.15.49:0/3903328093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff9419be80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff900117e0 tx=0x7eff90011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.215+0000 7eff8affd700 1 -- 172.21.15.49:0/3903328093 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff9001a430 con 0x7eff94109470 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.215+0000 7eff9c246700 1 -- 172.21.15.49:0/3903328093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff9419c650 con 0x7eff94109470 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.215+0000 7eff8affd700 1 -- 172.21.15.49:0/3903328093 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff9001aab0 con 0x7eff94109470 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.215+0000 7eff8affd700 1 -- 172.21.15.49:0/3903328093 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff90024a60 con 0x7eff94109470 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.215+0000 7eff9c246700 1 -- 172.21.15.49:0/3903328093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff9419cab0 con 0x7eff94109470 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.216+0000 7eff8affd700 1 -- 172.21.15.49:0/3903328093 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7eff90024d10 con 0x7eff94109470 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.216+0000 7eff9c246700 1 -- 172.21.15.49:0/3903328093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff940639e0 con 0x7eff94109470 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.216+0000 7eff8affd700 1 --2- 172.21.15.49:0/3903328093 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7eff80039180 0x7eff8003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:36.465 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.216+0000 7eff8affd700 1 -- 172.21.15.49:0/3903328093 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7eff9004d8b0 con 0x7eff94109470 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.216+0000 7eff997e1700 1 --2- 172.21.15.49:0/3903328093 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7eff80039180 0x7eff8003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.217+0000 7eff997e1700 1 --2- 172.21.15.49:0/3903328093 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7eff80039180 0x7eff8003b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7eff84006fd0 tx=0x7eff84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.221+0000 7eff8affd700 1 -- 172.21.15.49:0/3903328093 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7eff9002cdf0 con 0x7eff94109470 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:34.494+0000 7eff9c246700 1 -- 172.21.15.49:0/3903328093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7eff9419f7b0 con 0x7eff94109470 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.464+0000 7eff8affd700 1 -- 172.21.15.49:0/3903328093 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7eff9000fdd0 con 0x7eff94109470 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.466+0000 7eff8affd700 1 -- 172.21.15.49:0/3903328093 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mgrmap(e 9) ==== 46307+0+0 (secure 0 0 0) 0x7eff9004fd10 con 0x7eff94109470 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.467+0000 7eff9c246700 1 -- 172.21.15.49:0/3903328093 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7eff80039180 msgr2=0x7eff8003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.467+0000 7eff9c246700 1 --2- 172.21.15.49:0/3903328093 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7eff80039180 0x7eff8003b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7eff84006fd0 tx=0x7eff84006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.467+0000 7eff9c246700 1 -- 172.21.15.49:0/3903328093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 msgr2=0x7eff9419be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.467+0000 7eff9c246700 1 --2- 172.21.15.49:0/3903328093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff9419be80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff900117e0 tx=0x7eff90011ce0 comp rx=0 tx=0).stop 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.468+0000 7eff9c246700 1 -- 172.21.15.49:0/3903328093 shutdown_connections 2024-03-26T17:52:36.466 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.468+0000 7eff9c246700 1 --2- 172.21.15.49:0/3903328093 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7eff80039180 0x7eff8003b640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:36.467 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.468+0000 7eff9c246700 1 --2- 172.21.15.49:0/3903328093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff94109470 0x7eff9419be80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:36.467 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.468+0000 7eff9c246700 1 -- 172.21.15.49:0/3903328093 >> 172.21.15.49:0/3903328093 conn(0x7eff94103630 msgr2=0x7eff94104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:36.467 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.468+0000 7eff9c246700 1 -- 172.21.15.49:0/3903328093 shutdown_connections 2024-03-26T17:52:36.467 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:35.468+0000 7eff9c246700 1 -- 172.21.15.49:0/3903328093 wait complete. 2024-03-26T17:52:36.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:36 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3903328093' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-26T17:52:36.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:36 smithi049 ceph-mon[32331]: mgrmap e9: x(active, since 20s) 2024-03-26T17:52:37.075 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:36 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:36.834+0000 7f3574cff200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T17:52:37.075 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:36 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:36.937+0000 7f3574cff200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T17:52:37.076 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:37.085+0000 7f3574cff200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T17:52:37.456 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:37.185+0000 7f3574cff200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T17:52:37.456 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:37.276+0000 7f3574cff200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T17:52:37.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:37 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3301353319' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T17:52:37.746 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:37.551+0000 7f3574cff200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T17:52:37.746 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:37.666+0000 7f3574cff200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T17:52:37.746 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:37.755+0000 7f3574cff200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T17:52:37.870 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:52:37.870 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-26T17:52:37.870 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T17:52:37.870 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T17:52:37.870 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T17:52:37.870 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:52:37.870 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.911+0000 7fe5fb176700 1 Processor -- start 2024-03-26T17:52:37.870 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.911+0000 7fe5fb176700 1 -- start start 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.911+0000 7fe5fb176700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f4108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.912+0000 7fe5fb176700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f4108950 con 0x7fe5f4107ff0 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.912+0000 7fe5f8f12700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f4108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.912+0000 7fe5f8f12700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f4108410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42400/0 (socket says 172.21.15.49:42400) 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.912+0000 7fe5f8f12700 1 -- 172.21.15.49:0/4099361111 learned_addr learned my addr 172.21.15.49:0/4099361111 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.912+0000 7fe5f8f12700 1 -- 172.21.15.49:0/4099361111 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f4109060 con 0x7fe5f4107ff0 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.912+0000 7fe5f8f12700 1 --2- 172.21.15.49:0/4099361111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f4108410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80098a0 tx=0x7fe5e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cb219080e54c42 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.912+0000 7fe5f37fe700 1 -- 172.21.15.49:0/4099361111 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5e8004160 con 0x7fe5f4107ff0 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.913+0000 7fe5f37fe700 1 -- 172.21.15.49:0/4099361111 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5e8004300 con 0x7fe5f4107ff0 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.913+0000 7fe5f37fe700 1 -- 172.21.15.49:0/4099361111 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5e8011b20 con 0x7fe5f4107ff0 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.913+0000 7fe5fb176700 1 -- 172.21.15.49:0/4099361111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 msgr2=0x7fe5f4108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:37.871 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.913+0000 7fe5fb176700 1 --2- 172.21.15.49:0/4099361111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f4108410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80098a0 tx=0x7fe5e8009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.913+0000 7fe5fb176700 1 -- 172.21.15.49:0/4099361111 shutdown_connections 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.913+0000 7fe5fb176700 1 --2- 172.21.15.49:0/4099361111 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f4108410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.913+0000 7fe5fb176700 1 -- 172.21.15.49:0/4099361111 >> 172.21.15.49:0/4099361111 conn(0x7fe5f4103610 msgr2=0x7fe5f4105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.913+0000 7fe5fb176700 1 -- 172.21.15.49:0/4099361111 shutdown_connections 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.913+0000 7fe5fb176700 1 -- 172.21.15.49:0/4099361111 wait complete. 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.914+0000 7fe5fb176700 1 Processor -- start 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.914+0000 7fe5fb176700 1 -- start start 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.914+0000 7fe5fb176700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.914+0000 7fe5fb176700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f410c4a0 con 0x7fe5f4107ff0 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.914+0000 7fe5f8f12700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.914+0000 7fe5f8f12700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f419be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42416/0 (socket says 172.21.15.49:42416) 2024-03-26T17:52:37.872 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.914+0000 7fe5f8f12700 1 -- 172.21.15.49:0/3301353319 learned_addr learned my addr 172.21.15.49:0/3301353319 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.915+0000 7fe5f8f12700 1 -- 172.21.15.49:0/3301353319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f419c350 con 0x7fe5f4107ff0 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.915+0000 7fe5f8f12700 1 --2- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f419be10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80116a0 tx=0x7fe5e8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.915+0000 7fe5f1ffb700 1 -- 172.21.15.49:0/3301353319 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5e8003ac0 con 0x7fe5f4107ff0 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.915+0000 7fe5fb176700 1 -- 172.21.15.49:0/3301353319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f419c5e0 con 0x7fe5f4107ff0 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.915+0000 7fe5f1ffb700 1 -- 172.21.15.49:0/3301353319 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5e8003c60 con 0x7fe5f4107ff0 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.915+0000 7fe5f1ffb700 1 -- 172.21.15.49:0/3301353319 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5e801a7d0 con 0x7fe5f4107ff0 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.915+0000 7fe5fb176700 1 -- 172.21.15.49:0/3301353319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f419ca40 con 0x7fe5f4107ff0 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.916+0000 7fe5f1ffb700 1 -- 172.21.15.49:0/3301353319 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) ==== 46307+0+0 (secure 0 0 0) 0x7fe5e801aa80 con 0x7fe5f4107ff0 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.916+0000 7fe5fb176700 1 -- 172.21.15.49:0/3301353319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f40694b0 con 0x7fe5f4107ff0 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.916+0000 7fe5f1ffb700 1 --2- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fe5e40391d0 0x7fe5e403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:37.873 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.916+0000 7fe5f1ffb700 1 -- 172.21.15.49:0/3301353319 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe5e804d930 con 0x7fe5f4107ff0 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.916+0000 7fe5f3fff700 1 -- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fe5e40391d0 msgr2=0x7fe5e403b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4249557516 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.916+0000 7fe5f3fff700 1 --2- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fe5e40391d0 0x7fe5e403b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:36.925+0000 7fe5f1ffb700 1 -- 172.21.15.49:0/3301353319 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe5e8011e20 con 0x7fe5f4107ff0 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.117+0000 7fe5f3fff700 1 -- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fe5e40391d0 msgr2=0x7fe5e403b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4249557516 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.117+0000 7fe5f3fff700 1 --2- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fe5e40391d0 0x7fe5e403b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.191+0000 7fe5fb176700 1 -- 172.21.15.49:0/3301353319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe5f419f3f0 con 0x7fe5f4107ff0 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.191+0000 7fe5f1ffb700 1 -- 172.21.15.49:0/3301353319 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fe5e800bc40 con 0x7fe5f4107ff0 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.193+0000 7fe5fb176700 1 -- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fe5e40391d0 msgr2=0x7fe5e403b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.193+0000 7fe5fb176700 1 --2- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fe5e40391d0 0x7fe5e403b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.193+0000 7fe5fb176700 1 -- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 msgr2=0x7fe5f419be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:37.874 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.193+0000 7fe5fb176700 1 --2- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f419be10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80116a0 tx=0x7fe5e8011c10 comp rx=0 tx=0).stop 2024-03-26T17:52:37.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.193+0000 7fe5fb176700 1 -- 172.21.15.49:0/3301353319 shutdown_connections 2024-03-26T17:52:37.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.193+0000 7fe5fb176700 1 --2- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fe5e40391d0 0x7fe5e403b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:37.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.193+0000 7fe5fb176700 1 --2- 172.21.15.49:0/3301353319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5f4107ff0 0x7fe5f419be10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:37.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.193+0000 7fe5fb176700 1 -- 172.21.15.49:0/3301353319 >> 172.21.15.49:0/3301353319 conn(0x7fe5f4103610 msgr2=0x7fe5f4104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:37.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.193+0000 7fe5fb176700 1 -- 172.21.15.49:0/3301353319 shutdown_connections 2024-03-26T17:52:37.875 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:37.193+0000 7fe5fb176700 1 -- 172.21.15.49:0/3301353319 wait complete. 2024-03-26T17:52:37.875 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2024-03-26T17:52:37.875 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 9... 2024-03-26T17:52:38.003 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:37.854+0000 7f3574cff200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T17:52:38.260 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:38 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:38.040+0000 7f3574cff200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T17:52:39.188 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:39.061+0000 7f3574cff200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T17:52:39.189 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:39.198+0000 7f3574cff200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T17:52:39.832 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:39.544+0000 7f3574cff200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T17:52:39.832 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:39.636+0000 7f3574cff200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T17:52:39.832 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:39.842+0000 7f3574cff200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:52:40.260 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:40 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:40.036+0000 7f3574cff200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T17:52:40.594 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:40 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:40.335+0000 7f3574cff200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:52:40.594 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:40 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:40.603+0000 7f3574cff200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T17:52:41.010 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:40 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:40.850+0000 7f3574cff200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T17:52:42.010 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:41 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:41.688+0000 7f3574cff200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T17:52:42.598 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:42.280+0000 7f3574cff200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T17:52:42.599 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:42.382+0000 7f3574cff200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T17:52:42.599 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T17:52:42.608+0000 7f3574cff200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T17:52:43.010 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: Active manager daemon x restarted 2024-03-26T17:52:43.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: Activating manager daemon x 2024-03-26T17:52:43.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: osdmap e3: 0 total, 0 up, 0 in 2024-03-26T17:52:43.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: mgrmap e10: x(active, starting, since 0.00332684s) 2024-03-26T17:52:43.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:52:43.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T17:52:43.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T17:52:43.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T17:52:43.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T17:52:43.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: Manager daemon x is now available 2024-03-26T17:52:43.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:44.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T17:52:44.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T17:52:44.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:43 smithi049 ceph-mon[32331]: mgrmap e11: x(active, since 1.0064s) 2024-03-26T17:52:44.245 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:52:44.245 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.301+0000 7fd872878700 1 Processor -- start 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.301+0000 7fd872878700 1 -- start start 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.301+0000 7fd872878700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.301+0000 7fd872878700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c1077a0 con 0x7fd86c106e40 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.301+0000 7fd86bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.301+0000 7fd86bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42432/0 (socket says 172.21.15.49:42432) 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.301+0000 7fd86bfff700 1 -- 172.21.15.49:0/3853775162 learned_addr learned my addr 172.21.15.49:0/3853775162 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.302+0000 7fd86bfff700 1 -- 172.21.15.49:0/3853775162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c107fa0 con 0x7fd86c106e40 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.302+0000 7fd86bfff700 1 --2- 172.21.15.49:0/3853775162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c107260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0098a0 tx=0x7fd85c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b9fafcd3351dcb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.302+0000 7fd86affd700 1 -- 172.21.15.49:0/3853775162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd85c004160 con 0x7fd86c106e40 2024-03-26T17:52:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.302+0000 7fd86affd700 1 -- 172.21.15.49:0/3853775162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85c004300 con 0x7fd86c106e40 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.302+0000 7fd86affd700 1 -- 172.21.15.49:0/3853775162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd85c011b20 con 0x7fd86c106e40 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.303+0000 7fd872878700 1 -- 172.21.15.49:0/3853775162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 msgr2=0x7fd86c107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.303+0000 7fd872878700 1 --2- 172.21.15.49:0/3853775162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c107260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0098a0 tx=0x7fd85c009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.303+0000 7fd872878700 1 -- 172.21.15.49:0/3853775162 shutdown_connections 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.303+0000 7fd872878700 1 --2- 172.21.15.49:0/3853775162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c107260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.303+0000 7fd872878700 1 -- 172.21.15.49:0/3853775162 >> 172.21.15.49:0/3853775162 conn(0x7fd86c102420 msgr2=0x7fd86c104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.303+0000 7fd872878700 1 -- 172.21.15.49:0/3853775162 shutdown_connections 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.303+0000 7fd872878700 1 -- 172.21.15.49:0/3853775162 wait complete. 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.304+0000 7fd872878700 1 Processor -- start 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.304+0000 7fd872878700 1 -- start start 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.304+0000 7fd872878700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.247 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.304+0000 7fd872878700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c10b3e0 con 0x7fd86c106e40 2024-03-26T17:52:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.304+0000 7fd86bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.304+0000 7fd86bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c19be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42438/0 (socket says 172.21.15.49:42438) 2024-03-26T17:52:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.304+0000 7fd86bfff700 1 -- 172.21.15.49:0/1944296896 learned_addr learned my addr 172.21.15.49:0/1944296896 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.305+0000 7fd86bfff700 1 -- 172.21.15.49:0/1944296896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c19c370 con 0x7fd86c106e40 2024-03-26T17:52:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.305+0000 7fd86bfff700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c19be30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0117e0 tx=0x7fd85c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.305+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd85c003ba0 con 0x7fd86c106e40 2024-03-26T17:52:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.305+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85c01a910 con 0x7fd86c106e40 2024-03-26T17:52:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.305+0000 7fd872878700 1 -- 172.21.15.49:0/1944296896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86c19c600 con 0x7fd86c106e40 2024-03-26T17:52:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.305+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd85c0248f0 con 0x7fd86c106e40 2024-03-26T17:52:44.248 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.305+0000 7fd872878700 1 -- 172.21.15.49:0/1944296896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86c19ca60 con 0x7fd86c106e40 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.306+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 9) ==== 46307+0+0 (secure 0 0 0) 0x7fd85c022020 con 0x7fd86c106e40 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.306+0000 7fd8697fa700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 0x7fd85403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.306+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 --> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd85403bdf0 con 0x7fd854039220 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.306+0000 7fd86b7fe700 1 -- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 msgr2=0x7fd85403b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4249557516 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.306+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd85c04d770 con 0x7fd86c106e40 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.306+0000 7fd86b7fe700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 0x7fd85403b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.507+0000 7fd86b7fe700 1 -- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 msgr2=0x7fd85403b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4249557516 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.507+0000 7fd86b7fe700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 0x7fd85403b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.908+0000 7fd86b7fe700 1 -- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 msgr2=0x7fd85403b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4249557516 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:38.908+0000 7fd86b7fe700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 0x7fd85403b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:39.709+0000 7fd86b7fe700 1 -- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 msgr2=0x7fd85403b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4249557516 2024-03-26T17:52:44.249 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:39.709+0000 7fd86b7fe700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 0x7fd85403b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:41.310+0000 7fd86b7fe700 1 -- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 msgr2=0x7fd85403b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/4249557516 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:41.310+0000 7fd86b7fe700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 0x7fd85403b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:42.623+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 10) ==== 46074+0+0 (secure 0 0 0) 0x7fd85c02d3f0 con 0x7fd86c106e40 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:42.623+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 msgr2=0x7fd85403b6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:42.623+0000 7fd8697fa700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 0x7fd85403b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.625+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 11) ==== 46201+0+0 (secure 0 0 0) 0x7fd85c04f1f0 con 0x7fd86c106e40 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.625+0000 7fd8697fa700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd85403cca0 0x7fd85403f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.625+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd85403bdf0 con 0x7fd85403cca0 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.625+0000 7fd86b7fe700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd85403cca0 0x7fd85403f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.626+0000 7fd86b7fe700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd85403cca0 0x7fd85403f090 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd860003a10 tx=0x7fd860009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.626+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fd85403bdf0 con 0x7fd85403cca0 2024-03-26T17:52:44.250 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.633+0000 7fd872878700 1 -- 172.21.15.49:0/1944296896 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd86c0694b0 con 0x7fd85403cca0 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.633+0000 7fd8697fa700 1 -- 172.21.15.49:0/1944296896 <== mgr.14152 v2:172.21.15.49:6800/3127469495 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd86c0694b0 con 0x7fd85403cca0 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 -- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd85403cca0 msgr2=0x7fd85403f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd85403cca0 0x7fd85403f090 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd860003a10 tx=0x7fd860009270 comp rx=0 tx=0).stop 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 -- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 msgr2=0x7fd86c19be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c19be30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0117e0 tx=0x7fd85c011ba0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 -- 172.21.15.49:0/1944296896 shutdown_connections 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd85403cca0 0x7fd85403f090 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd86c106e40 0x7fd86c19be30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 --2- 172.21.15.49:0/1944296896 >> [v2:172.21.15.49:6800/4249557516,v1:172.21.15.49:6801/4249557516] conn(0x7fd854039220 0x7fd85403b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 -- 172.21.15.49:0/1944296896 >> 172.21.15.49:0/1944296896 conn(0x7fd86c102420 msgr2=0x7fd86c102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 -- 172.21.15.49:0/1944296896 shutdown_connections 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:43.634+0000 7fd872878700 1 -- 172.21.15.49:0/1944296896 wait complete. 2024-03-26T17:52:44.251 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 9 is available 2024-03-26T17:52:44.252 INFO:teuthology.orchestra.run.smithi049.stdout:Generating a dashboard self-signed certificate... 2024-03-26T17:52:44.968 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:44 smithi049 ceph-mon[32331]: [26/Mar/2024:17:52:43] ENGINE Bus STARTING 2024-03-26T17:52:44.968 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:44 smithi049 ceph-mon[32331]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T17:52:44.968 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:44 smithi049 ceph-mon[32331]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T17:52:44.968 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:44 smithi049 ceph-mon[32331]: [26/Mar/2024:17:52:43] ENGINE Serving on https://172.21.15.49:7150 2024-03-26T17:52:44.968 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:44 smithi049 ceph-mon[32331]: [26/Mar/2024:17:52:43] ENGINE Serving on http://172.21.15.49:8765 2024-03-26T17:52:44.968 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:44 smithi049 ceph-mon[32331]: [26/Mar/2024:17:52:43] ENGINE Bus STARTED 2024-03-26T17:52:45.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-26T17:52:45.568 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.681+0000 7f9ef9254700 1 Processor -- start 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.681+0000 7f9ef9254700 1 -- start start 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.681+0000 7f9ef9254700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.681+0000 7f9ef9254700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4107830 con 0x7f9ef4106e40 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.682+0000 7f9ef2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.682+0000 7f9ef2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53726/0 (socket says 172.21.15.49:53726) 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.682+0000 7f9ef2d9d700 1 -- 172.21.15.49:0/4156791327 learned_addr learned my addr 172.21.15.49:0/4156791327 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.682+0000 7f9ef2d9d700 1 -- 172.21.15.49:0/4156791327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef4107ff0 con 0x7f9ef4106e40 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.682+0000 7f9ef2d9d700 1 --2- 172.21.15.49:0/4156791327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4107260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400bb40 tx=0x7f9ee400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ba281dca58307 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.682+0000 7f9ef1d9b700 1 -- 172.21.15.49:0/4156791327 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee400dbe0 con 0x7f9ef4106e40 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.682+0000 7f9ef1d9b700 1 -- 172.21.15.49:0/4156791327 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee4004510 con 0x7f9ef4106e40 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.682+0000 7f9ef1d9b700 1 -- 172.21.15.49:0/4156791327 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee4010470 con 0x7f9ef4106e40 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.683+0000 7f9ef9254700 1 -- 172.21.15.49:0/4156791327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 msgr2=0x7f9ef4107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.569 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.683+0000 7f9ef9254700 1 --2- 172.21.15.49:0/4156791327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4107260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400bb40 tx=0x7f9ee400be50 comp rx=0 tx=0).stop 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.683+0000 7f9ef9254700 1 -- 172.21.15.49:0/4156791327 shutdown_connections 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.683+0000 7f9ef9254700 1 --2- 172.21.15.49:0/4156791327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4107260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.683+0000 7f9ef9254700 1 -- 172.21.15.49:0/4156791327 >> 172.21.15.49:0/4156791327 conn(0x7f9ef4102400 msgr2=0x7f9ef4104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.683+0000 7f9ef9254700 1 -- 172.21.15.49:0/4156791327 shutdown_connections 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.683+0000 7f9ef9254700 1 -- 172.21.15.49:0/4156791327 wait complete. 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.684+0000 7f9ef9254700 1 Processor -- start 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.684+0000 7f9ef9254700 1 -- start start 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.684+0000 7f9ef9254700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.684+0000 7f9ef9254700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef407b790 con 0x7f9ef4106e40 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.684+0000 7f9ef2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.684+0000 7f9ef2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53732/0 (socket says 172.21.15.49:53732) 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.684+0000 7f9ef2d9d700 1 -- 172.21.15.49:0/1960509627 learned_addr learned my addr 172.21.15.49:0/1960509627 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:45.570 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.684+0000 7f9ef2d9d700 1 -- 172.21.15.49:0/1960509627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef419a2e0 con 0x7f9ef4106e40 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.685+0000 7f9ef2d9d700 1 --2- 172.21.15.49:0/1960509627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4199da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4003bb0 tx=0x7f9ee40040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.685+0000 7f9edbfff700 1 -- 172.21.15.49:0/1960509627 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee4027070 con 0x7f9ef4106e40 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.685+0000 7f9edbfff700 1 -- 172.21.15.49:0/1960509627 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee400de20 con 0x7f9ef4106e40 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.685+0000 7f9ef9254700 1 -- 172.21.15.49:0/1960509627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef419a570 con 0x7f9ef4106e40 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.685+0000 7f9edbfff700 1 -- 172.21.15.49:0/1960509627 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee4022ba0 con 0x7f9ef4106e40 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.685+0000 7f9ef9254700 1 -- 172.21.15.49:0/1960509627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef419d220 con 0x7f9ef4106e40 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.686+0000 7f9edbfff700 1 -- 172.21.15.49:0/1960509627 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 46307+0+0 (secure 0 0 0) 0x7f9ee402a070 con 0x7f9ef4106e40 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.686+0000 7f9ef9254700 1 -- 172.21.15.49:0/1960509627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee0005360 con 0x7f9ef4106e40 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.686+0000 7f9edbfff700 1 --2- 172.21.15.49:0/1960509627 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9edc039180 0x7f9edc03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.686+0000 7f9edbfff700 1 -- 172.21.15.49:0/1960509627 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9ee401f070 con 0x7f9ef4106e40 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.686+0000 7f9ef259c700 1 --2- 172.21.15.49:0/1960509627 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9edc039180 0x7f9edc03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.686+0000 7f9ef259c700 1 --2- 172.21.15.49:0/1960509627 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9edc039180 0x7f9edc03b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8006fd0 tx=0x7f9ee8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.691+0000 7f9edbfff700 1 -- 172.21.15.49:0/1960509627 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9ee4015070 con 0x7f9ef4106e40 2024-03-26T17:52:45.571 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.905+0000 7f9ef9254700 1 -- 172.21.15.49:0/1960509627 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f9ee0000bf0 con 0x7f9edc039180 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.977+0000 7f9edbfff700 1 -- 172.21.15.49:0/1960509627 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f9ee0000bf0 con 0x7f9edc039180 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.980+0000 7f9ef9254700 1 -- 172.21.15.49:0/1960509627 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9edc039180 msgr2=0x7f9edc03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.980+0000 7f9ef9254700 1 --2- 172.21.15.49:0/1960509627 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9edc039180 0x7f9edc03b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8006fd0 tx=0x7f9ee8006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.980+0000 7f9ef9254700 1 -- 172.21.15.49:0/1960509627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 msgr2=0x7f9ef4199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.980+0000 7f9ef9254700 1 --2- 172.21.15.49:0/1960509627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4199da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4003bb0 tx=0x7f9ee40040b0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.980+0000 7f9ef9254700 1 -- 172.21.15.49:0/1960509627 shutdown_connections 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.980+0000 7f9ef9254700 1 --2- 172.21.15.49:0/1960509627 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9edc039180 0x7f9edc03b640 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.980+0000 7f9ef9254700 1 --2- 172.21.15.49:0/1960509627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ef4106e40 0x7f9ef4199da0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.980+0000 7f9ef9254700 1 -- 172.21.15.49:0/1960509627 >> 172.21.15.49:0/1960509627 conn(0x7f9ef4102400 msgr2=0x7f9ef41907c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.980+0000 7f9ef9254700 1 -- 172.21.15.49:0/1960509627 shutdown_connections 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:44.980+0000 7f9ef9254700 1 -- 172.21.15.49:0/1960509627 wait complete. 2024-03-26T17:52:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial admin user... 2024-03-26T17:52:45.955 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:45 smithi049 ceph-mon[32331]: mgrmap e12: x(active, since 2s) 2024-03-26T17:52:45.955 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:45 smithi049 ceph-mon[32331]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:52:45.955 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:45.955 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$n5FiugythkeM68jqBvV1Jeee0.UsRts60oDn1aVMifbsaFRCtqLY2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711475566, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.002+0000 7effa8486700 1 Processor -- start 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.003+0000 7effa8486700 1 -- start start 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.003+0000 7effa8486700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa0107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.003+0000 7effa8486700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7effa01077a0 con 0x7effa0106e40 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.003+0000 7effa6222700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa0107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.003+0000 7effa6222700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa0107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53736/0 (socket says 172.21.15.49:53736) 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.003+0000 7effa6222700 1 -- 172.21.15.49:0/3363195975 learned_addr learned my addr 172.21.15.49:0/3363195975 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.003+0000 7effa6222700 1 -- 172.21.15.49:0/3363195975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa0107fa0 con 0x7effa0106e40 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.003+0000 7effa6222700 1 --2- 172.21.15.49:0/3363195975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa0107260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff940098a0 tx=0x7eff94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572138b02f9e147e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.004+0000 7effa5220700 1 -- 172.21.15.49:0/3363195975 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff94004160 con 0x7effa0106e40 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.004+0000 7effa5220700 1 -- 172.21.15.49:0/3363195975 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff94004300 con 0x7effa0106e40 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.004+0000 7effa5220700 1 -- 172.21.15.49:0/3363195975 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff94011b20 con 0x7effa0106e40 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.004+0000 7effa8486700 1 -- 172.21.15.49:0/3363195975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 msgr2=0x7effa0107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.004+0000 7effa8486700 1 --2- 172.21.15.49:0/3363195975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa0107260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff940098a0 tx=0x7eff94009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.004+0000 7effa8486700 1 -- 172.21.15.49:0/3363195975 shutdown_connections 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.004+0000 7effa8486700 1 --2- 172.21.15.49:0/3363195975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa0107260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.004+0000 7effa8486700 1 -- 172.21.15.49:0/3363195975 >> 172.21.15.49:0/3363195975 conn(0x7effa0102480 msgr2=0x7effa01048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.004+0000 7effa8486700 1 -- 172.21.15.49:0/3363195975 shutdown_connections 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.004+0000 7effa8486700 1 -- 172.21.15.49:0/3363195975 wait complete. 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.005+0000 7effa8486700 1 Processor -- start 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.005+0000 7effa8486700 1 -- start start 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.005+0000 7effa8486700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.005+0000 7effa8486700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7effa010b3e0 con 0x7effa0106e40 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.005+0000 7effa6222700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.005+0000 7effa6222700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa019e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53746/0 (socket says 172.21.15.49:53746) 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.005+0000 7effa6222700 1 -- 172.21.15.49:0/60339898 learned_addr learned my addr 172.21.15.49:0/60339898 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.006+0000 7effa6222700 1 -- 172.21.15.49:0/60339898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa019e5b0 con 0x7effa0106e40 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.006+0000 7effa6222700 1 --2- 172.21.15.49:0/60339898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa019e070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff940116a0 tx=0x7eff94011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.006+0000 7eff937fe700 1 -- 172.21.15.49:0/60339898 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff94003a40 con 0x7effa0106e40 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.006+0000 7eff937fe700 1 -- 172.21.15.49:0/60339898 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff94003be0 con 0x7effa0106e40 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.006+0000 7effa8486700 1 -- 172.21.15.49:0/60339898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa019e840 con 0x7effa0106e40 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.006+0000 7eff937fe700 1 -- 172.21.15.49:0/60339898 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff9401a750 con 0x7effa0106e40 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.006+0000 7effa8486700 1 -- 172.21.15.49:0/60339898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa019eca0 con 0x7effa0106e40 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.007+0000 7eff937fe700 1 -- 172.21.15.49:0/60339898 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 46307+0+0 (secure 0 0 0) 0x7eff9401aa70 con 0x7effa0106e40 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.007+0000 7eff937fe700 1 --2- 172.21.15.49:0/60339898 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff8c039180 0x7eff8c03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.007+0000 7effa8486700 1 -- 172.21.15.49:0/60339898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa00694b0 con 0x7effa0106e40 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.007+0000 7eff937fe700 1 -- 172.21.15.49:0/60339898 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7eff9404d990 con 0x7effa0106e40 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.007+0000 7effa5a21700 1 --2- 172.21.15.49:0/60339898 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff8c039180 0x7eff8c03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.007+0000 7effa5a21700 1 --2- 172.21.15.49:0/60339898 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff8c039180 0x7eff8c03b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff9c006fd0 tx=0x7eff9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.012+0000 7eff937fe700 1 -- 172.21.15.49:0/60339898 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7eff94011e20 con 0x7effa0106e40 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.257+0000 7effa8486700 1 -- 172.21.15.49:0/60339898 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7effa0198120 con 0x7eff8c039180 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.503+0000 7eff937fe700 1 -- 172.21.15.49:0/60339898 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7effa0198120 con 0x7eff8c039180 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.505+0000 7effa8486700 1 -- 172.21.15.49:0/60339898 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff8c039180 msgr2=0x7eff8c03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.505+0000 7effa8486700 1 --2- 172.21.15.49:0/60339898 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff8c039180 0x7eff8c03b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff9c006fd0 tx=0x7eff9c006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.505+0000 7effa8486700 1 -- 172.21.15.49:0/60339898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 msgr2=0x7effa019e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.505+0000 7effa8486700 1 --2- 172.21.15.49:0/60339898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa019e070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff940116a0 tx=0x7eff94011ba0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.505+0000 7effa8486700 1 -- 172.21.15.49:0/60339898 shutdown_connections 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.505+0000 7effa8486700 1 --2- 172.21.15.49:0/60339898 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff8c039180 0x7eff8c03b640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.505+0000 7effa8486700 1 --2- 172.21.15.49:0/60339898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effa0106e40 0x7effa019e070 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.505+0000 7effa8486700 1 -- 172.21.15.49:0/60339898 >> 172.21.15.49:0/60339898 conn(0x7effa0102480 msgr2=0x7effa01031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.506+0000 7effa8486700 1 -- 172.21.15.49:0/60339898 shutdown_connections 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:46.506+0000 7effa8486700 1 -- 172.21.15.49:0/60339898 wait complete. 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi049.stdout:Fetching dashboard port number... 2024-03-26T17:52:47.260 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:47.260 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:47 smithi049 ceph-mon[32331]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:52:47.260 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:48.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 8443 2024-03-26T17:52:48.371 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.600+0000 7f471b666700 1 Processor -- start 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.601+0000 7f471b666700 1 -- start start 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.601+0000 7f471b666700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471410ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.601+0000 7f471b666700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f471407c670 con 0x7f4714108980 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.601+0000 7f4719402700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471410ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.601+0000 7f4719402700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471410ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53748/0 (socket says 172.21.15.49:53748) 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.601+0000 7f4719402700 1 -- 172.21.15.49:0/51207804 learned_addr learned my addr 172.21.15.49:0/51207804 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.601+0000 7f4719402700 1 -- 172.21.15.49:0/51207804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471410b2b0 con 0x7f4714108980 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.602+0000 7f4719402700 1 --2- 172.21.15.49:0/51207804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471410ad70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47100098a0 tx=0x7f4710009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d364f4b14468f923 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.602+0000 7f470bfff700 1 -- 172.21.15.49:0/51207804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4710004160 con 0x7f4714108980 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.602+0000 7f470bfff700 1 -- 172.21.15.49:0/51207804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4710004300 con 0x7f4714108980 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.602+0000 7f470bfff700 1 -- 172.21.15.49:0/51207804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4710011b20 con 0x7f4714108980 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.603+0000 7f471b666700 1 -- 172.21.15.49:0/51207804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 msgr2=0x7f471410ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:48.372 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.603+0000 7f471b666700 1 --2- 172.21.15.49:0/51207804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471410ad70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47100098a0 tx=0x7f4710009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.603+0000 7f471b666700 1 -- 172.21.15.49:0/51207804 shutdown_connections 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.603+0000 7f471b666700 1 --2- 172.21.15.49:0/51207804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471410ad70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.603+0000 7f471b666700 1 -- 172.21.15.49:0/51207804 >> 172.21.15.49:0/51207804 conn(0x7f4714102490 msgr2=0x7f47141048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.603+0000 7f471b666700 1 -- 172.21.15.49:0/51207804 shutdown_connections 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.603+0000 7f471b666700 1 -- 172.21.15.49:0/51207804 wait complete. 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.603+0000 7f471b666700 1 Processor -- start 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.603+0000 7f471b666700 1 -- start start 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.604+0000 7f471b666700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.604+0000 7f471b666700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f471410c320 con 0x7f4714108980 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.604+0000 7f4719402700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.604+0000 7f4719402700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471419bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53764/0 (socket says 172.21.15.49:53764) 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.604+0000 7f4719402700 1 -- 172.21.15.49:0/1421302614 learned_addr learned my addr 172.21.15.49:0/1421302614 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.604+0000 7f4719402700 1 -- 172.21.15.49:0/1421302614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471419c300 con 0x7f4714108980 2024-03-26T17:52:48.373 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.604+0000 7f4719402700 1 --2- 172.21.15.49:0/1421302614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471419bdc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47100117e0 tx=0x7f4710011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.604+0000 7f470a7fc700 1 -- 172.21.15.49:0/1421302614 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4710003bf0 con 0x7f4714108980 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.604+0000 7f470a7fc700 1 -- 172.21.15.49:0/1421302614 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f471001a910 con 0x7f4714108980 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.604+0000 7f471b666700 1 -- 172.21.15.49:0/1421302614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471419c590 con 0x7f4714108980 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.605+0000 7f470a7fc700 1 -- 172.21.15.49:0/1421302614 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4710024970 con 0x7f4714108980 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.605+0000 7f471b666700 1 -- 172.21.15.49:0/1421302614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f471419c9f0 con 0x7f4714108980 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.605+0000 7f470a7fc700 1 -- 172.21.15.49:0/1421302614 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 46307+0+0 (secure 0 0 0) 0x7f4710022020 con 0x7f4714108980 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.605+0000 7f470a7fc700 1 --2- 172.21.15.49:0/1421302614 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f47000391d0 0x7f470003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.606+0000 7f470a7fc700 1 -- 172.21.15.49:0/1421302614 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f471004e580 con 0x7f4714108980 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.606+0000 7f4718c01700 1 --2- 172.21.15.49:0/1421302614 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f47000391d0 0x7f470003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.606+0000 7f471b666700 1 -- 172.21.15.49:0/1421302614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47140639e0 con 0x7f4714108980 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.606+0000 7f4718c01700 1 --2- 172.21.15.49:0/1421302614 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f47000391d0 0x7f470003b690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4704006fd0 tx=0x7f4704006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.611+0000 7f470a7fc700 1 -- 172.21.15.49:0/1421302614 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4710011e20 con 0x7f4714108980 2024-03-26T17:52:48.374 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.820+0000 7f471b666700 1 -- 172.21.15.49:0/1421302614 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f471419f3c0 con 0x7f4714108980 2024-03-26T17:52:48.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.821+0000 7f470a7fc700 1 -- 172.21.15.49:0/1421302614 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f471000ba30 con 0x7f4714108980 2024-03-26T17:52:48.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.823+0000 7f471b666700 1 -- 172.21.15.49:0/1421302614 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f47000391d0 msgr2=0x7f470003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:48.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.823+0000 7f471b666700 1 --2- 172.21.15.49:0/1421302614 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f47000391d0 0x7f470003b690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4704006fd0 tx=0x7f4704006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:48.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.823+0000 7f471b666700 1 -- 172.21.15.49:0/1421302614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 msgr2=0x7f471419bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:48.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.823+0000 7f471b666700 1 --2- 172.21.15.49:0/1421302614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471419bdc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47100117e0 tx=0x7f4710011c10 comp rx=0 tx=0).stop 2024-03-26T17:52:48.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.823+0000 7f471b666700 1 -- 172.21.15.49:0/1421302614 shutdown_connections 2024-03-26T17:52:48.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.823+0000 7f471b666700 1 --2- 172.21.15.49:0/1421302614 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f47000391d0 0x7f470003b690 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:48.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.823+0000 7f471b666700 1 --2- 172.21.15.49:0/1421302614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4714108980 0x7f471419bdc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:48.375 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.823+0000 7f471b666700 1 -- 172.21.15.49:0/1421302614 >> 172.21.15.49:0/1421302614 conn(0x7f4714102490 msgr2=0x7f4714102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:48.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.823+0000 7f471b666700 1 -- 172.21.15.49:0/1421302614 shutdown_connections 2024-03-26T17:52:48.376 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:47.823+0000 7f471b666700 1 -- 172.21.15.49:0/1421302614 wait complete. 2024-03-26T17:52:48.379 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T17:52:48.379 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2024-03-26T17:52:48.387 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T17:52:48.387 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2024-03-26T17:52:48.387 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2024-03-26T17:52:48.387 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-26T17:52:48.389 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph Dashboard is now available at: 2024-03-26T17:52:48.389 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:52:48.389 INFO:teuthology.orchestra.run.smithi049.stdout: URL: https://smithi049.front.sepia.ceph.com:8443/ 2024-03-26T17:52:48.389 INFO:teuthology.orchestra.run.smithi049.stdout: User: admin 2024-03-26T17:52:48.389 INFO:teuthology.orchestra.run.smithi049.stdout: Password: m1gugh8nyf 2024-03-26T17:52:48.389 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:52:48.390 INFO:teuthology.orchestra.run.smithi049.stdout:Saving cluster configuration to /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config directory 2024-03-26T17:52:48.390 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling autotune for osd_memory_target 2024-03-26T17:52:48.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:48 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1421302614' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-26T17:52:48.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:48 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:50.260 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:50 smithi049 ceph-mon[32331]: mgrmap e13: x(active, since 6s) 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.988+0000 7fa0c580a700 1 Processor -- start 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.988+0000 7fa0c580a700 1 -- start start 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.988+0000 7fa0c580a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c010adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.988+0000 7fa0c580a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c007bdc0 con 0x7fa0c01089d0 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.988+0000 7fa0beffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c010adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.988+0000 7fa0beffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c010adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33560/0 (socket says 172.21.15.49:33560) 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.988+0000 7fa0beffd700 1 -- 172.21.15.49:0/2802672982 learned_addr learned my addr 172.21.15.49:0/2802672982 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.989+0000 7fa0beffd700 1 -- 172.21.15.49:0/2802672982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c010b300 con 0x7fa0c01089d0 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.989+0000 7fa0beffd700 1 --2- 172.21.15.49:0/2802672982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c010adc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00098a0 tx=0x7fa0b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92aaeb42dbcdac5b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.989+0000 7fa0bdffb700 1 -- 172.21.15.49:0/2802672982 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0004160 con 0x7fa0c01089d0 2024-03-26T17:52:50.552 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.989+0000 7fa0bdffb700 1 -- 172.21.15.49:0/2802672982 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b0004300 con 0x7fa0c01089d0 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.989+0000 7fa0bdffb700 1 -- 172.21.15.49:0/2802672982 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0011b20 con 0x7fa0c01089d0 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.990+0000 7fa0c580a700 1 -- 172.21.15.49:0/2802672982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 msgr2=0x7fa0c010adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.990+0000 7fa0c580a700 1 --2- 172.21.15.49:0/2802672982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c010adc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00098a0 tx=0x7fa0b0009bb0 comp rx=0 tx=0).stop 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.990+0000 7fa0c580a700 1 -- 172.21.15.49:0/2802672982 shutdown_connections 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.990+0000 7fa0c580a700 1 --2- 172.21.15.49:0/2802672982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c010adc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.990+0000 7fa0c580a700 1 -- 172.21.15.49:0/2802672982 >> 172.21.15.49:0/2802672982 conn(0x7fa0c01023e0 msgr2=0x7fa0c0104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.990+0000 7fa0c580a700 1 -- 172.21.15.49:0/2802672982 shutdown_connections 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.990+0000 7fa0c580a700 1 -- 172.21.15.49:0/2802672982 wait complete. 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.991+0000 7fa0c580a700 1 Processor -- start 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.991+0000 7fa0c580a700 1 -- start start 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.991+0000 7fa0c580a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.991+0000 7fa0c580a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010c300 con 0x7fa0c01089d0 2024-03-26T17:52:50.553 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.992+0000 7fa0beffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.992+0000 7fa0beffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c019bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33562/0 (socket says 172.21.15.49:33562) 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.992+0000 7fa0beffd700 1 -- 172.21.15.49:0/2988362143 learned_addr learned my addr 172.21.15.49:0/2988362143 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.992+0000 7fa0beffd700 1 -- 172.21.15.49:0/2988362143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c019c2e0 con 0x7fa0c01089d0 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.992+0000 7fa0beffd700 1 --2- 172.21.15.49:0/2988362143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c019bda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00117e0 tx=0x7fa0b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.992+0000 7fa0c4808700 1 -- 172.21.15.49:0/2988362143 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0003bf0 con 0x7fa0c01089d0 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.992+0000 7fa0c580a700 1 -- 172.21.15.49:0/2988362143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c019c5d0 con 0x7fa0c01089d0 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.992+0000 7fa0c4808700 1 -- 172.21.15.49:0/2988362143 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b001a910 con 0x7fa0c01089d0 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.992+0000 7fa0c4808700 1 -- 172.21.15.49:0/2988362143 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0024970 con 0x7fa0c01089d0 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.992+0000 7fa0c580a700 1 -- 172.21.15.49:0/2988362143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c019f260 con 0x7fa0c01089d0 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.993+0000 7fa0c4808700 1 -- 172.21.15.49:0/2988362143 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 12) ==== 46307+0+0 (secure 0 0 0) 0x7fa0b0022020 con 0x7fa0c01089d0 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.993+0000 7fa0c580a700 1 -- 172.21.15.49:0/2988362143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c00639e0 con 0x7fa0c01089d0 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.994+0000 7fa0c4808700 1 --2- 172.21.15.49:0/2988362143 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa0a80391d0 0x7fa0a803b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:50.554 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.994+0000 7fa0c4808700 1 -- 172.21.15.49:0/2988362143 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa0b004d680 con 0x7fa0c01089d0 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.994+0000 7fa0be7fc700 1 --2- 172.21.15.49:0/2988362143 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa0a80391d0 0x7fa0a803b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.994+0000 7fa0be7fc700 1 --2- 172.21.15.49:0/2988362143 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa0a80391d0 0x7fa0a803b690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006fd0 tx=0x7fa0b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:48.999+0000 7fa0c4808700 1 -- 172.21.15.49:0/2988362143 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa0b0011e20 con 0x7fa0c01089d0 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.143+0000 7fa0c4808700 1 -- 172.21.15.49:0/2988362143 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fa0b0024cd0 con 0x7fa0c01089d0 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.208+0000 7fa0c580a700 1 -- 172.21.15.49:0/2988362143 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fa0c019f430 con 0x7fa0c01089d0 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.208+0000 7fa0c4808700 1 -- 172.21.15.49:0/2988362143 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fa0b0019c70 con 0x7fa0c01089d0 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.211+0000 7fa0c580a700 1 -- 172.21.15.49:0/2988362143 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa0a80391d0 msgr2=0x7fa0a803b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.211+0000 7fa0c580a700 1 --2- 172.21.15.49:0/2988362143 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa0a80391d0 0x7fa0a803b690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006fd0 tx=0x7fa0b4006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.211+0000 7fa0c580a700 1 -- 172.21.15.49:0/2988362143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 msgr2=0x7fa0c019bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.211+0000 7fa0c580a700 1 --2- 172.21.15.49:0/2988362143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c019bda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00117e0 tx=0x7fa0b0011c10 comp rx=0 tx=0).stop 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.211+0000 7fa0c580a700 1 -- 172.21.15.49:0/2988362143 shutdown_connections 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.211+0000 7fa0c580a700 1 --2- 172.21.15.49:0/2988362143 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa0a80391d0 0x7fa0a803b690 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:50.555 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.211+0000 7fa0c580a700 1 --2- 172.21.15.49:0/2988362143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0c01089d0 0x7fa0c019bda0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:50.556 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.211+0000 7fa0c580a700 1 -- 172.21.15.49:0/2988362143 >> 172.21.15.49:0/2988362143 conn(0x7fa0c01023e0 msgr2=0x7fa0c0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:50.556 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.211+0000 7fa0c580a700 1 -- 172.21.15.49:0/2988362143 shutdown_connections 2024-03-26T17:52:50.556 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:49.211+0000 7fa0c580a700 1 -- 172.21.15.49:0/2988362143 wait complete. 2024-03-26T17:52:51.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.950+0000 7fb245f60700 1 Processor -- start 2024-03-26T17:52:51.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.950+0000 7fb245f60700 1 -- start start 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.951+0000 7fb245f60700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240106650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.951+0000 7fb245f60700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb240106c20 con 0x7fb240106230 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.951+0000 7fb23f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240106650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.951+0000 7fb23f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240106650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33566/0 (socket says 172.21.15.49:33566) 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.951+0000 7fb23f7fe700 1 -- 172.21.15.49:0/398430999 learned_addr learned my addr 172.21.15.49:0/398430999 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.951+0000 7fb23f7fe700 1 -- 172.21.15.49:0/398430999 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2401091c0 con 0x7fb240106230 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.951+0000 7fb23f7fe700 1 --2- 172.21.15.49:0/398430999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240106650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb230009e80 tx=0x7fb23000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7240a7621c5945c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.951+0000 7fb23e7fc700 1 -- 172.21.15.49:0/398430999 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb230017070 con 0x7fb240106230 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.951+0000 7fb23e7fc700 1 -- 172.21.15.49:0/398430999 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb23000b8c0 con 0x7fb240106230 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.951+0000 7fb23e7fc700 1 -- 172.21.15.49:0/398430999 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2300045c0 con 0x7fb240106230 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.952+0000 7fb245f60700 1 -- 172.21.15.49:0/398430999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 msgr2=0x7fb240106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:51.808 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.952+0000 7fb245f60700 1 --2- 172.21.15.49:0/398430999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240106650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb230009e80 tx=0x7fb23000b330 comp rx=0 tx=0).stop 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.952+0000 7fb245f60700 1 -- 172.21.15.49:0/398430999 shutdown_connections 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.952+0000 7fb245f60700 1 --2- 172.21.15.49:0/398430999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240106650 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.952+0000 7fb245f60700 1 -- 172.21.15.49:0/398430999 >> 172.21.15.49:0/398430999 conn(0x7fb240101b60 msgr2=0x7fb240103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.952+0000 7fb245f60700 1 -- 172.21.15.49:0/398430999 shutdown_connections 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.952+0000 7fb245f60700 1 -- 172.21.15.49:0/398430999 wait complete. 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.953+0000 7fb245f60700 1 Processor -- start 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.953+0000 7fb245f60700 1 -- start start 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.953+0000 7fb245f60700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240108a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.953+0000 7fb245f60700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb24010c570 con 0x7fb240106230 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.953+0000 7fb23f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240108a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.953+0000 7fb23f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240108a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33580/0 (socket says 172.21.15.49:33580) 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.953+0000 7fb23f7fe700 1 -- 172.21.15.49:0/3851317471 learned_addr learned my addr 172.21.15.49:0/3851317471 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.954+0000 7fb23f7fe700 1 -- 172.21.15.49:0/3851317471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb240108f80 con 0x7fb240106230 2024-03-26T17:52:51.809 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.954+0000 7fb23f7fe700 1 --2- 172.21.15.49:0/3851317471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240108a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb23000bca0 tx=0x7fb23000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.954+0000 7fb23cff9700 1 -- 172.21.15.49:0/3851317471 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb230020020 con 0x7fb240106230 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.954+0000 7fb23cff9700 1 -- 172.21.15.49:0/3851317471 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb230003d90 con 0x7fb240106230 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.954+0000 7fb23cff9700 1 -- 172.21.15.49:0/3851317471 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb230017040 con 0x7fb240106230 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.954+0000 7fb245f60700 1 -- 172.21.15.49:0/3851317471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb240107170 con 0x7fb240106230 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.954+0000 7fb245f60700 1 -- 172.21.15.49:0/3851317471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2401076b0 con 0x7fb240106230 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.955+0000 7fb23cff9700 1 -- 172.21.15.49:0/3851317471 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fb230023020 con 0x7fb240106230 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.955+0000 7fb245f60700 1 -- 172.21.15.49:0/3851317471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2400639e0 con 0x7fb240106230 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.955+0000 7fb23cff9700 1 --2- 172.21.15.49:0/3851317471 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb22c039270 0x7fb22c03b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.955+0000 7fb23cff9700 1 -- 172.21.15.49:0/3851317471 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb23000f050 con 0x7fb240106230 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.955+0000 7fb23effd700 1 --2- 172.21.15.49:0/3851317471 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb22c039270 0x7fb22c03b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.956+0000 7fb23effd700 1 --2- 172.21.15.49:0/3851317471 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb22c039270 0x7fb22c03b730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb228006fd0 tx=0x7fb228006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:51.810 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:50.960+0000 7fb23cff9700 1 -- 172.21.15.49:0/3851317471 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb23002a020 con 0x7fb240106230 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.277+0000 7fb245f60700 1 -- 172.21.15.49:0/3851317471 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb240107880 con 0x7fb240106230 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.280+0000 7fb23cff9700 1 -- 172.21.15.49:0/3851317471 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fb23001b530 con 0x7fb240106230 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.282+0000 7fb245f60700 1 -- 172.21.15.49:0/3851317471 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb22c039270 msgr2=0x7fb22c03b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.282+0000 7fb245f60700 1 --2- 172.21.15.49:0/3851317471 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb22c039270 0x7fb22c03b730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb228006fd0 tx=0x7fb228006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.282+0000 7fb245f60700 1 -- 172.21.15.49:0/3851317471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 msgr2=0x7fb240108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.282+0000 7fb245f60700 1 --2- 172.21.15.49:0/3851317471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240108a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb23000bca0 tx=0x7fb23000bd80 comp rx=0 tx=0).stop 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.282+0000 7fb245f60700 1 -- 172.21.15.49:0/3851317471 shutdown_connections 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.282+0000 7fb245f60700 1 --2- 172.21.15.49:0/3851317471 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb22c039270 0x7fb22c03b730 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.282+0000 7fb245f60700 1 --2- 172.21.15.49:0/3851317471 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb240106230 0x7fb240108a40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.282+0000 7fb245f60700 1 -- 172.21.15.49:0/3851317471 >> 172.21.15.49:0/3851317471 conn(0x7fb240101b60 msgr2=0x7fb240102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.282+0000 7fb245f60700 1 -- 172.21.15.49:0/3851317471 shutdown_connections 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr 2024-03-26T17:52:51.282+0000 7fb245f60700 1 -- 172.21.15.49:0/3851317471 wait complete. 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-26T17:52:51.811 INFO:teuthology.orchestra.run.smithi049.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout:Or, if you are only running a single cluster on this host: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: ceph telemetry on 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout:For more information see: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:52:51.812 INFO:teuthology.orchestra.run.smithi049.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-26T17:52:51.813 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:52:51.813 INFO:teuthology.orchestra.run.smithi049.stdout:Bootstrap complete. 2024-03-26T17:52:51.850 INFO:tasks.cephadm:Fetching config... 2024-03-26T17:52:51.851 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:52:51.851 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-26T17:52:51.867 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-26T17:52:51.867 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:52:51.867 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-26T17:52:51.923 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-26T17:52:51.923 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:52:51.923 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-26T17:52:51.994 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-26T17:52:51.994 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:52:51.994 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-26T17:52:52.049 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-26T17:52:52.049 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSkR9ilOPddk77g+new7Gm/JuT9LZI1Ur9jDrIcW2YhHbPfa3ZePyA38nZVGG3UxF8N7H07V1w9ckXPb3bI2Tx7vQYpTb4WemiVl+MNsri7kFfMbyErMR/xXuszw6rQ2DURMNdsg7wogRivb9BkrRl02H8ehtyCuR8G0XZNRzF60oav1d1B7aKg10Wy2Arj/PKzNywbYeXBKUUkfmvzfuTD9/BkU/6KATE2UKiyyb78W/R++6mhtnz2ywOXEYoX8v7R89q1vidtvSRCOAAeanwO0x56CWgJQ5wftmznrj0ScsE2Sa1B/NAiGSa3P9xtliqTJ4rs8NJRsDqtDTvEmBsuQKIh/aEywEnxvUYBUMz16jdx551LBgp/bSJSun9aCHCBd3W9JFeD0PiWHTfZ85kJS60vtKvCSBHOBnRiHo2gl1jh6MiUP6Jw9/gMgbYUC94lf6YUGWNSBUR11AEYes0YniFQK4xjAVeVf/ArefPw8glBgt26gGvQOKE4zBUN6c= ceph-5a3632a4-eb99-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T17:52:52.149 INFO:teuthology.orchestra.run.smithi049.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSkR9ilOPddk77g+new7Gm/JuT9LZI1Ur9jDrIcW2YhHbPfa3ZePyA38nZVGG3UxF8N7H07V1w9ckXPb3bI2Tx7vQYpTb4WemiVl+MNsri7kFfMbyErMR/xXuszw6rQ2DURMNdsg7wogRivb9BkrRl02H8ehtyCuR8G0XZNRzF60oav1d1B7aKg10Wy2Arj/PKzNywbYeXBKUUkfmvzfuTD9/BkU/6KATE2UKiyyb78W/R++6mhtnz2ywOXEYoX8v7R89q1vidtvSRCOAAeanwO0x56CWgJQ5wftmznrj0ScsE2Sa1B/NAiGSa3P9xtliqTJ4rs8NJRsDqtDTvEmBsuQKIh/aEywEnxvUYBUMz16jdx551LBgp/bSJSun9aCHCBd3W9JFeD0PiWHTfZ85kJS60vtKvCSBHOBnRiHo2gl1jh6MiUP6Jw9/gMgbYUC94lf6YUGWNSBUR11AEYes0YniFQK4xjAVeVf/ArefPw8glBgt26gGvQOKE4zBUN6c= ceph-5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:52:52.166 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSkR9ilOPddk77g+new7Gm/JuT9LZI1Ur9jDrIcW2YhHbPfa3ZePyA38nZVGG3UxF8N7H07V1w9ckXPb3bI2Tx7vQYpTb4WemiVl+MNsri7kFfMbyErMR/xXuszw6rQ2DURMNdsg7wogRivb9BkrRl02H8ehtyCuR8G0XZNRzF60oav1d1B7aKg10Wy2Arj/PKzNywbYeXBKUUkfmvzfuTD9/BkU/6KATE2UKiyyb78W/R++6mhtnz2ywOXEYoX8v7R89q1vidtvSRCOAAeanwO0x56CWgJQ5wftmznrj0ScsE2Sa1B/NAiGSa3P9xtliqTJ4rs8NJRsDqtDTvEmBsuQKIh/aEywEnxvUYBUMz16jdx551LBgp/bSJSun9aCHCBd3W9JFeD0PiWHTfZ85kJS60vtKvCSBHOBnRiHo2gl1jh6MiUP6Jw9/gMgbYUC94lf6YUGWNSBUR11AEYes0YniFQK4xjAVeVf/ArefPw8glBgt26gGvQOKE4zBUN6c= ceph-5a3632a4-eb99-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T17:52:52.213 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSkR9ilOPddk77g+new7Gm/JuT9LZI1Ur9jDrIcW2YhHbPfa3ZePyA38nZVGG3UxF8N7H07V1w9ckXPb3bI2Tx7vQYpTb4WemiVl+MNsri7kFfMbyErMR/xXuszw6rQ2DURMNdsg7wogRivb9BkrRl02H8ehtyCuR8G0XZNRzF60oav1d1B7aKg10Wy2Arj/PKzNywbYeXBKUUkfmvzfuTD9/BkU/6KATE2UKiyyb78W/R++6mhtnz2ywOXEYoX8v7R89q1vidtvSRCOAAeanwO0x56CWgJQ5wftmznrj0ScsE2Sa1B/NAiGSa3P9xtliqTJ4rs8NJRsDqtDTvEmBsuQKIh/aEywEnxvUYBUMz16jdx551LBgp/bSJSun9aCHCBd3W9JFeD0PiWHTfZ85kJS60vtKvCSBHOBnRiHo2gl1jh6MiUP6Jw9/gMgbYUC94lf6YUGWNSBUR11AEYes0YniFQK4xjAVeVf/ArefPw8glBgt26gGvQOKE4zBUN6c= ceph-5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:52:52.228 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSkR9ilOPddk77g+new7Gm/JuT9LZI1Ur9jDrIcW2YhHbPfa3ZePyA38nZVGG3UxF8N7H07V1w9ckXPb3bI2Tx7vQYpTb4WemiVl+MNsri7kFfMbyErMR/xXuszw6rQ2DURMNdsg7wogRivb9BkrRl02H8ehtyCuR8G0XZNRzF60oav1d1B7aKg10Wy2Arj/PKzNywbYeXBKUUkfmvzfuTD9/BkU/6KATE2UKiyyb78W/R++6mhtnz2ywOXEYoX8v7R89q1vidtvSRCOAAeanwO0x56CWgJQ5wftmznrj0ScsE2Sa1B/NAiGSa3P9xtliqTJ4rs8NJRsDqtDTvEmBsuQKIh/aEywEnxvUYBUMz16jdx551LBgp/bSJSun9aCHCBd3W9JFeD0PiWHTfZ85kJS60vtKvCSBHOBnRiHo2gl1jh6MiUP6Jw9/gMgbYUC94lf6YUGWNSBUR11AEYes0YniFQK4xjAVeVf/ArefPw8glBgt26gGvQOKE4zBUN6c= ceph-5a3632a4-eb99-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T17:52:52.278 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSkR9ilOPddk77g+new7Gm/JuT9LZI1Ur9jDrIcW2YhHbPfa3ZePyA38nZVGG3UxF8N7H07V1w9ckXPb3bI2Tx7vQYpTb4WemiVl+MNsri7kFfMbyErMR/xXuszw6rQ2DURMNdsg7wogRivb9BkrRl02H8ehtyCuR8G0XZNRzF60oav1d1B7aKg10Wy2Arj/PKzNywbYeXBKUUkfmvzfuTD9/BkU/6KATE2UKiyyb78W/R++6mhtnz2ywOXEYoX8v7R89q1vidtvSRCOAAeanwO0x56CWgJQ5wftmznrj0ScsE2Sa1B/NAiGSa3P9xtliqTJ4rs8NJRsDqtDTvEmBsuQKIh/aEywEnxvUYBUMz16jdx551LBgp/bSJSun9aCHCBd3W9JFeD0PiWHTfZ85kJS60vtKvCSBHOBnRiHo2gl1jh6MiUP6Jw9/gMgbYUC94lf6YUGWNSBUR11AEYes0YniFQK4xjAVeVf/ArefPw8glBgt26gGvQOKE4zBUN6c= ceph-5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:52:52.295 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-26T17:52:52.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:52 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3851317471' entity='client.admin' 2024-03-26T17:52:52.545 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:52:54.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.924+0000 7fec8ce54700 1 -- 172.21.15.49:0/2413305109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88101e00 msgr2=0x7fec88102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:54.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.924+0000 7fec8ce54700 1 --2- 172.21.15.49:0/2413305109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88101e00 0x7fec88102220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec7800b0a0 tx=0x7fec7800b3b0 comp rx=0 tx=0).stop 2024-03-26T17:52:54.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.924+0000 7fec8ce54700 1 -- 172.21.15.49:0/2413305109 shutdown_connections 2024-03-26T17:52:54.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.924+0000 7fec8ce54700 1 --2- 172.21.15.49:0/2413305109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88101e00 0x7fec88102220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:54.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.924+0000 7fec8ce54700 1 -- 172.21.15.49:0/2413305109 >> 172.21.15.49:0/2413305109 conn(0x7fec880fd3a0 msgr2=0x7fec880ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:54.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.924+0000 7fec8ce54700 1 -- 172.21.15.49:0/2413305109 shutdown_connections 2024-03-26T17:52:54.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.924+0000 7fec8ce54700 1 -- 172.21.15.49:0/2413305109 wait complete. 2024-03-26T17:52:54.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.925+0000 7fec8ce54700 1 Processor -- start 2024-03-26T17:52:54.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.925+0000 7fec8ce54700 1 -- start start 2024-03-26T17:52:54.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.925+0000 7fec8ce54700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88101e00 0x7fec88199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:54.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.925+0000 7fec8ce54700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec88107e10 con 0x7fec88101e00 2024-03-26T17:52:54.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.925+0000 7fec8659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88101e00 0x7fec88199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:54.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.925+0000 7fec8659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88101e00 0x7fec88199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33612/0 (socket says 172.21.15.49:33612) 2024-03-26T17:52:54.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.925+0000 7fec8659c700 1 -- 172.21.15.49:0/229318933 learned_addr learned my addr 172.21.15.49:0/229318933 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:54.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.925+0000 7fec8659c700 1 -- 172.21.15.49:0/229318933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec88199b80 con 0x7fec88101e00 2024-03-26T17:52:54.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.926+0000 7fec8659c700 1 --2- 172.21.15.49:0/229318933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88101e00 0x7fec88199640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec78007890 tx=0x7fec7800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:54.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.926+0000 7fec7f7fe700 1 -- 172.21.15.49:0/229318933 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec78017040 con 0x7fec88101e00 2024-03-26T17:52:54.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.926+0000 7fec7f7fe700 1 -- 172.21.15.49:0/229318933 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec78022070 con 0x7fec88101e00 2024-03-26T17:52:54.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.926+0000 7fec8ce54700 1 -- 172.21.15.49:0/229318933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec88199e10 con 0x7fec88101e00 2024-03-26T17:52:54.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.926+0000 7fec7f7fe700 1 -- 172.21.15.49:0/229318933 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec7801d5b0 con 0x7fec88101e00 2024-03-26T17:52:54.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.926+0000 7fec8ce54700 1 -- 172.21.15.49:0/229318933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8819cb00 con 0x7fec88101e00 2024-03-26T17:52:54.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.927+0000 7fec7f7fe700 1 -- 172.21.15.49:0/229318933 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fec7801d860 con 0x7fec88101e00 2024-03-26T17:52:54.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.927+0000 7fec7f7fe700 1 --2- 172.21.15.49:0/229318933 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec74039260 0x7fec7403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:54.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.927+0000 7fec7f7fe700 1 -- 172.21.15.49:0/229318933 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fec7800f050 con 0x7fec88101e00 2024-03-26T17:52:54.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.927+0000 7fec8ce54700 1 -- 172.21.15.49:0/229318933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec88108590 con 0x7fec88101e00 2024-03-26T17:52:54.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.927+0000 7fec85d9b700 1 --2- 172.21.15.49:0/229318933 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec74039260 0x7fec7403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:54.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.927+0000 7fec85d9b700 1 --2- 172.21.15.49:0/229318933 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec74039260 0x7fec7403b720 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec70006fd0 tx=0x7fec70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:54.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:54.932+0000 7fec7f7fe700 1 -- 172.21.15.49:0/229318933 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fec78007e20 con 0x7fec88101e00 2024-03-26T17:52:55.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.142+0000 7fec8ce54700 1 -- 172.21.15.49:0/229318933 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fec8802dc00 con 0x7fec88101e00 2024-03-26T17:52:55.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.144+0000 7fec7f7fe700 1 -- 172.21.15.49:0/229318933 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fec780263c0 con 0x7fec88101e00 2024-03-26T17:52:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.146+0000 7fec8ce54700 1 -- 172.21.15.49:0/229318933 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec74039260 msgr2=0x7fec7403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.146+0000 7fec8ce54700 1 --2- 172.21.15.49:0/229318933 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec74039260 0x7fec7403b720 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec70006fd0 tx=0x7fec70006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.146+0000 7fec8ce54700 1 -- 172.21.15.49:0/229318933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88101e00 msgr2=0x7fec88199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.146+0000 7fec8ce54700 1 --2- 172.21.15.49:0/229318933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88101e00 0x7fec88199640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec78007890 tx=0x7fec7800bd40 comp rx=0 tx=0).stop 2024-03-26T17:52:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.147+0000 7fec8ce54700 1 -- 172.21.15.49:0/229318933 shutdown_connections 2024-03-26T17:52:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.147+0000 7fec8ce54700 1 --2- 172.21.15.49:0/229318933 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec74039260 0x7fec7403b720 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.147+0000 7fec8ce54700 1 --2- 172.21.15.49:0/229318933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec88101e00 0x7fec88199640 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.147+0000 7fec8ce54700 1 -- 172.21.15.49:0/229318933 >> 172.21.15.49:0/229318933 conn(0x7fec880fd3a0 msgr2=0x7fec880fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:55.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.147+0000 7fec8ce54700 1 -- 172.21.15.49:0/229318933 shutdown_connections 2024-03-26T17:52:55.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:55.147+0000 7fec8ce54700 1 -- 172.21.15.49:0/229318933 wait complete. 2024-03-26T17:52:55.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:55 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:55.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:55 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:55.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:55 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:52:55.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:55 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:55.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:55 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:55.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:55 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:55.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:55 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/229318933' entity='client.admin' 2024-03-26T17:52:55.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:55 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:55.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:55 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:55.770 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-26T17:52:55.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-26T17:52:56.047 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:52:58.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.266+0000 7fdd131ea700 1 -- 172.21.15.49:0/3706174004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c102870 msgr2=0x7fdd0c104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.266+0000 7fdd131ea700 1 --2- 172.21.15.49:0/3706174004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c102870 0x7fdd0c104c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009910 tx=0x7fdd00009c20 comp rx=0 tx=0).stop 2024-03-26T17:52:58.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.266+0000 7fdd131ea700 1 -- 172.21.15.49:0/3706174004 shutdown_connections 2024-03-26T17:52:58.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.266+0000 7fdd131ea700 1 --2- 172.21.15.49:0/3706174004 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c102870 0x7fdd0c104c60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.266+0000 7fdd131ea700 1 -- 172.21.15.49:0/3706174004 >> 172.21.15.49:0/3706174004 conn(0x7fdd0c0fc280 msgr2=0x7fdd0c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:58.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.266+0000 7fdd131ea700 1 -- 172.21.15.49:0/3706174004 shutdown_connections 2024-03-26T17:52:58.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.266+0000 7fdd131ea700 1 -- 172.21.15.49:0/3706174004 wait complete. 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.267+0000 7fdd131ea700 1 Processor -- start 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.267+0000 7fdd131ea700 1 -- start start 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.267+0000 7fdd131ea700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c102870 0x7fdd0c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.267+0000 7fdd131ea700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c107e10 con 0x7fdd0c102870 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.267+0000 7fdd10f86700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c102870 0x7fdd0c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.267+0000 7fdd10f86700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c102870 0x7fdd0c199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33638/0 (socket says 172.21.15.49:33638) 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.267+0000 7fdd10f86700 1 -- 172.21.15.49:0/285271370 learned_addr learned my addr 172.21.15.49:0/285271370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.267+0000 7fdd10f86700 1 -- 172.21.15.49:0/285271370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c199bc0 con 0x7fdd0c102870 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.268+0000 7fdd10f86700 1 --2- 172.21.15.49:0/285271370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c102870 0x7fdd0c199680 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd00004fa0 tx=0x7fdd00005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.268+0000 7fdd09ffb700 1 -- 172.21.15.49:0/285271370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd00027070 con 0x7fdd0c102870 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.268+0000 7fdd09ffb700 1 -- 172.21.15.49:0/285271370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0000be40 con 0x7fdd0c102870 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.268+0000 7fdd131ea700 1 -- 172.21.15.49:0/285271370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c199eb0 con 0x7fdd0c102870 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.268+0000 7fdd09ffb700 1 -- 172.21.15.49:0/285271370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd0000f460 con 0x7fdd0c102870 2024-03-26T17:52:58.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.268+0000 7fdd131ea700 1 -- 172.21.15.49:0/285271370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c19cb40 con 0x7fdd0c102870 2024-03-26T17:52:58.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.269+0000 7fdd09ffb700 1 -- 172.21.15.49:0/285271370 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fdd0002a070 con 0x7fdd0c102870 2024-03-26T17:52:58.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.269+0000 7fdd131ea700 1 -- 172.21.15.49:0/285271370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c108590 con 0x7fdd0c102870 2024-03-26T17:52:58.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.269+0000 7fdd09ffb700 1 --2- 172.21.15.49:0/285271370 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdcfc039280 0x7fdcfc03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.269+0000 7fdd09ffb700 1 -- 172.21.15.49:0/285271370 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdd0001e030 con 0x7fdd0c102870 2024-03-26T17:52:58.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.269+0000 7fdd0bfff700 1 --2- 172.21.15.49:0/285271370 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdcfc039280 0x7fdcfc03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.270+0000 7fdd0bfff700 1 --2- 172.21.15.49:0/285271370 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdcfc039280 0x7fdcfc03b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8006fd0 tx=0x7fdcf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:58.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.274+0000 7fdd09ffb700 1 -- 172.21.15.49:0/285271370 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdd00013070 con 0x7fdd0c102870 2024-03-26T17:52:58.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.481+0000 7fdd131ea700 1 -- 172.21.15.49:0/285271370 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fdd0c02ddf0 con 0x7fdcfc039280 2024-03-26T17:52:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.485+0000 7fdd09ffb700 1 -- 172.21.15.49:0/285271370 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdd0c02ddf0 con 0x7fdcfc039280 2024-03-26T17:52:58.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.487+0000 7fdd131ea700 1 -- 172.21.15.49:0/285271370 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdcfc039280 msgr2=0x7fdcfc03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.487+0000 7fdd131ea700 1 --2- 172.21.15.49:0/285271370 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdcfc039280 0x7fdcfc03b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8006fd0 tx=0x7fdcf8006e40 comp rx=0 tx=0).stop 2024-03-26T17:52:58.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.487+0000 7fdd131ea700 1 -- 172.21.15.49:0/285271370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c102870 msgr2=0x7fdd0c199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.487+0000 7fdd131ea700 1 --2- 172.21.15.49:0/285271370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c102870 0x7fdd0c199680 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd00004fa0 tx=0x7fdd00005080 comp rx=0 tx=0).stop 2024-03-26T17:52:58.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.487+0000 7fdd131ea700 1 -- 172.21.15.49:0/285271370 shutdown_connections 2024-03-26T17:52:58.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.487+0000 7fdd131ea700 1 --2- 172.21.15.49:0/285271370 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdcfc039280 0x7fdcfc03b740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.487+0000 7fdd131ea700 1 --2- 172.21.15.49:0/285271370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd0c102870 0x7fdd0c199680 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.487+0000 7fdd131ea700 1 -- 172.21.15.49:0/285271370 >> 172.21.15.49:0/285271370 conn(0x7fdd0c0fc280 msgr2=0x7fdd0c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:58.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.487+0000 7fdd131ea700 1 -- 172.21.15.49:0/285271370 shutdown_connections 2024-03-26T17:52:58.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:52:58.487+0000 7fdd131ea700 1 -- 172.21.15.49:0/285271370 wait complete. 2024-03-26T17:52:59.275 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi115 2024-03-26T17:52:59.275 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:52:59.276 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.conf 2024-03-26T17:52:59.293 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:52:59.294 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:52:59.350 INFO:tasks.cephadm:Adding host smithi115 to orchestrator... 2024-03-26T17:52:59.351 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch host add smithi115 2024-03-26T17:52:59.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:59 smithi049 ceph-mon[32331]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:52:59.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:59.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:59.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:52:59.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:52:59.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:59 smithi049 ceph-mon[32331]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-26T17:52:59.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:59.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:59.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:52:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:52:59.597 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:53:00.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:00 smithi049 ceph-mon[32331]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:00.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:00 smithi049 ceph-mon[32331]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:00.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:00 smithi049 ceph-mon[32331]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:53:01.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.132+0000 7f5380606700 1 -- 172.21.15.49:0/3987162552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5378101ff0 msgr2=0x7f53781023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:01.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.132+0000 7f5380606700 1 --2- 172.21.15.49:0/3987162552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5378101ff0 0x7f53781023d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5374009910 tx=0x7f5374009c20 comp rx=0 tx=0).stop 2024-03-26T17:53:01.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.132+0000 7f5380606700 1 -- 172.21.15.49:0/3987162552 shutdown_connections 2024-03-26T17:53:01.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.132+0000 7f5380606700 1 --2- 172.21.15.49:0/3987162552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5378101ff0 0x7f53781023d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.132+0000 7f5380606700 1 -- 172.21.15.49:0/3987162552 >> 172.21.15.49:0/3987162552 conn(0x7f53780fd8c0 msgr2=0x7f53780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.132+0000 7f5380606700 1 -- 172.21.15.49:0/3987162552 shutdown_connections 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.133+0000 7f5380606700 1 -- 172.21.15.49:0/3987162552 wait complete. 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.133+0000 7f5380606700 1 Processor -- start 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.133+0000 7f5380606700 1 -- start start 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.133+0000 7f5380606700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5378101ff0 0x7f537819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.133+0000 7f5380606700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f537810c3c0 con 0x7f5378101ff0 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.133+0000 7f537e3a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5378101ff0 0x7f537819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.133+0000 7f537e3a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5378101ff0 0x7f537819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46786/0 (socket says 172.21.15.49:46786) 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.133+0000 7f537e3a2700 1 -- 172.21.15.49:0/3132520806 learned_addr learned my addr 172.21.15.49:0/3132520806 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:53:01.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.134+0000 7f537e3a2700 1 -- 172.21.15.49:0/3132520806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537819db50 con 0x7f5378101ff0 2024-03-26T17:53:01.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.134+0000 7f537e3a2700 1 --2- 172.21.15.49:0/3132520806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5378101ff0 0x7f537819d610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5374006010 tx=0x7f5374005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:01.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.134+0000 7f536f7fe700 1 -- 172.21.15.49:0/3132520806 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5374026070 con 0x7f5378101ff0 2024-03-26T17:53:01.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.134+0000 7f5380606700 1 -- 172.21.15.49:0/3132520806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537819de40 con 0x7f5378101ff0 2024-03-26T17:53:01.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.134+0000 7f536f7fe700 1 -- 172.21.15.49:0/3132520806 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537400bbd0 con 0x7f5378101ff0 2024-03-26T17:53:01.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.134+0000 7f536f7fe700 1 -- 172.21.15.49:0/3132520806 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5374003910 con 0x7f5378101ff0 2024-03-26T17:53:01.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.134+0000 7f5380606700 1 -- 172.21.15.49:0/3132520806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5378197790 con 0x7f5378101ff0 2024-03-26T17:53:01.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.135+0000 7f536f7fe700 1 -- 172.21.15.49:0/3132520806 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f5374029070 con 0x7f5378101ff0 2024-03-26T17:53:01.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.135+0000 7f5380606700 1 -- 172.21.15.49:0/3132520806 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53781034d0 con 0x7f5378101ff0 2024-03-26T17:53:01.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.135+0000 7f536f7fe700 1 --2- 172.21.15.49:0/3132520806 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5364039220 0x7f536403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:01.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.135+0000 7f536f7fe700 1 -- 172.21.15.49:0/3132520806 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f537401e070 con 0x7f5378101ff0 2024-03-26T17:53:01.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.135+0000 7f537dba1700 1 --2- 172.21.15.49:0/3132520806 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5364039220 0x7f536403b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:01.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.136+0000 7f537dba1700 1 --2- 172.21.15.49:0/3132520806 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5364039220 0x7f536403b6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5368006fd0 tx=0x7f5368006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:01.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.140+0000 7f536f7fe700 1 -- 172.21.15.49:0/3132520806 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5374013070 con 0x7f5378101ff0 2024-03-26T17:53:01.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:01.348+0000 7f5380606700 1 -- 172.21.15.49:0/3132520806 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}) -- 0x7f5378198cc0 con 0x7f5364039220 2024-03-26T17:53:02.010 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:01 smithi049 ceph-mon[32331]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:53:03.010 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:02 smithi049 ceph-mon[32331]: Deploying cephadm binary to smithi115 2024-03-26T17:53:03.911 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:03 smithi049 ceph-mon[32331]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:05.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.641+0000 7f536f7fe700 1 -- 172.21.15.49:0/3132520806 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5378198cc0 con 0x7f5364039220 2024-03-26T17:53:05.632 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi115' with addr '172.21.15.115' 2024-03-26T17:53:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.643+0000 7f5380606700 1 -- 172.21.15.49:0/3132520806 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5364039220 msgr2=0x7f536403b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.643+0000 7f5380606700 1 --2- 172.21.15.49:0/3132520806 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5364039220 0x7f536403b6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5368006fd0 tx=0x7f5368006e40 comp rx=0 tx=0).stop 2024-03-26T17:53:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.643+0000 7f5380606700 1 -- 172.21.15.49:0/3132520806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5378101ff0 msgr2=0x7f537819d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.643+0000 7f5380606700 1 --2- 172.21.15.49:0/3132520806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5378101ff0 0x7f537819d610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5374006010 tx=0x7f5374005030 comp rx=0 tx=0).stop 2024-03-26T17:53:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.643+0000 7f5380606700 1 -- 172.21.15.49:0/3132520806 shutdown_connections 2024-03-26T17:53:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.643+0000 7f5380606700 1 --2- 172.21.15.49:0/3132520806 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5364039220 0x7f536403b6e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.643+0000 7f5380606700 1 --2- 172.21.15.49:0/3132520806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5378101ff0 0x7f537819d610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.643+0000 7f5380606700 1 -- 172.21.15.49:0/3132520806 >> 172.21.15.49:0/3132520806 conn(0x7f53780fd8c0 msgr2=0x7f5378104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.643+0000 7f5380606700 1 -- 172.21.15.49:0/3132520806 shutdown_connections 2024-03-26T17:53:05.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:05.643+0000 7f5380606700 1 -- 172.21.15.49:0/3132520806 wait complete. 2024-03-26T17:53:05.960 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:05 smithi049 ceph-mon[32331]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:05.960 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:05.960 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:06.202 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T17:53:06.449 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:53:07.431 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:07 smithi049 ceph-mon[32331]: Added host smithi115 2024-03-26T17:53:07.431 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:07 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:07.431 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:07 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:07.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.940+0000 7f9e3f774700 1 -- 172.21.15.49:0/2700369900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38100ad0 msgr2=0x7f9e38100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:07.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.940+0000 7f9e3f774700 1 --2- 172.21.15.49:0/2700369900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38100ad0 0x7f9e38100eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009910 tx=0x7f9e2c009c20 comp rx=0 tx=0).stop 2024-03-26T17:53:07.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.940+0000 7f9e3f774700 1 -- 172.21.15.49:0/2700369900 shutdown_connections 2024-03-26T17:53:07.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.940+0000 7f9e3f774700 1 --2- 172.21.15.49:0/2700369900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38100ad0 0x7f9e38100eb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:07.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.940+0000 7f9e3f774700 1 -- 172.21.15.49:0/2700369900 >> 172.21.15.49:0/2700369900 conn(0x7f9e380778f0 msgr2=0x7f9e380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:07.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.940+0000 7f9e3f774700 1 -- 172.21.15.49:0/2700369900 shutdown_connections 2024-03-26T17:53:07.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.940+0000 7f9e3f774700 1 -- 172.21.15.49:0/2700369900 wait complete. 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.940+0000 7f9e3f774700 1 Processor -- start 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.940+0000 7f9e3f774700 1 -- start start 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.940+0000 7f9e3f774700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38100ad0 0x7f9e3819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e3f774700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3810e610 con 0x7f9e38100ad0 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e3d510700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38100ad0 0x7f9e3819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e3d510700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38100ad0 0x7f9e3819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46792/0 (socket says 172.21.15.49:46792) 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e3d510700 1 -- 172.21.15.49:0/1721503915 learned_addr learned my addr 172.21.15.49:0/1721503915 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e3d510700 1 -- 172.21.15.49:0/1721503915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3819fc40 con 0x7f9e38100ad0 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e3d510700 1 --2- 172.21.15.49:0/1721503915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38100ad0 0x7f9e3819f700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c005850 tx=0x7f9e2c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e2a7fc700 1 -- 172.21.15.49:0/1721503915 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e2c026070 con 0x7f9e38100ad0 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e2a7fc700 1 -- 172.21.15.49:0/1721503915 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e2c00bc30 con 0x7f9e38100ad0 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e3f774700 1 -- 172.21.15.49:0/1721503915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3819fed0 con 0x7f9e38100ad0 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e2a7fc700 1 -- 172.21.15.49:0/1721503915 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e2c003a20 con 0x7f9e38100ad0 2024-03-26T17:53:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.941+0000 7f9e3f774700 1 -- 172.21.15.49:0/1721503915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e381998c0 con 0x7f9e38100ad0 2024-03-26T17:53:07.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.942+0000 7f9e2a7fc700 1 -- 172.21.15.49:0/1721503915 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f9e2c029070 con 0x7f9e38100ad0 2024-03-26T17:53:07.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.942+0000 7f9e3f774700 1 -- 172.21.15.49:0/1721503915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e3806a080 con 0x7f9e38100ad0 2024-03-26T17:53:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.942+0000 7f9e2a7fc700 1 --2- 172.21.15.49:0/1721503915 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9e24039280 0x7f9e2403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.942+0000 7f9e2a7fc700 1 -- 172.21.15.49:0/1721503915 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9e2c01e070 con 0x7f9e38100ad0 2024-03-26T17:53:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.942+0000 7f9e3cd0f700 1 --2- 172.21.15.49:0/1721503915 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9e24039280 0x7f9e2403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:07.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.943+0000 7f9e3cd0f700 1 --2- 172.21.15.49:0/1721503915 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9e24039280 0x7f9e2403b740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e34006fd0 tx=0x7f9e34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:07.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:07.947+0000 7f9e2a7fc700 1 -- 172.21.15.49:0/1721503915 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9e2c013070 con 0x7f9e38100ad0 2024-03-26T17:53:08.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.160+0000 7f9e3f774700 1 -- 172.21.15.49:0/1721503915 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e3819adf0 con 0x7f9e24039280 2024-03-26T17:53:08.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.161+0000 7f9e2a7fc700 1 -- 172.21.15.49:0/1721503915 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f9e3819adf0 con 0x7f9e24039280 2024-03-26T17:53:08.151 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:53:08.151 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}] 2024-03-26T17:53:08.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.163+0000 7f9e3f774700 1 -- 172.21.15.49:0/1721503915 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9e24039280 msgr2=0x7f9e2403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:08.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.163+0000 7f9e3f774700 1 --2- 172.21.15.49:0/1721503915 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9e24039280 0x7f9e2403b740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e34006fd0 tx=0x7f9e34006e40 comp rx=0 tx=0).stop 2024-03-26T17:53:08.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.163+0000 7f9e3f774700 1 -- 172.21.15.49:0/1721503915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38100ad0 msgr2=0x7f9e3819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:08.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.163+0000 7f9e3f774700 1 --2- 172.21.15.49:0/1721503915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38100ad0 0x7f9e3819f700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c005850 tx=0x7f9e2c005030 comp rx=0 tx=0).stop 2024-03-26T17:53:08.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.163+0000 7f9e3f774700 1 -- 172.21.15.49:0/1721503915 shutdown_connections 2024-03-26T17:53:08.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.163+0000 7f9e3f774700 1 --2- 172.21.15.49:0/1721503915 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9e24039280 0x7f9e2403b740 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:08.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.163+0000 7f9e3f774700 1 --2- 172.21.15.49:0/1721503915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9e38100ad0 0x7f9e3819f700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:08.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.163+0000 7f9e3f774700 1 -- 172.21.15.49:0/1721503915 >> 172.21.15.49:0/1721503915 conn(0x7f9e380778f0 msgr2=0x7f9e380fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:08.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.163+0000 7f9e3f774700 1 -- 172.21.15.49:0/1721503915 shutdown_connections 2024-03-26T17:53:08.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:08.163+0000 7f9e3f774700 1 -- 172.21.15.49:0/1721503915 wait complete. 2024-03-26T17:53:08.446 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:08 smithi049 ceph-mon[32331]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:08.446 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:08 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:08.748 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2024-03-26T17:53:08.749 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:53:08.749 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2024-03-26T17:53:08.767 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:53:08.768 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:08.825 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2024-03-26T17:53:08.825 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch host add smithi171 2024-03-26T17:53:09.070 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:53:09.399 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:09 smithi049 ceph-mon[32331]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:53:10.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:10 smithi049 ceph-mon[32331]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:10.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.597+0000 7f80fdf89700 1 -- 172.21.15.49:0/2840819381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f8102fa0 msgr2=0x7f80f8103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:10.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.597+0000 7f80fdf89700 1 --2- 172.21.15.49:0/2840819381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f8102fa0 0x7f80f8103380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009910 tx=0x7f80e8009c20 comp rx=0 tx=0).stop 2024-03-26T17:53:10.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.598+0000 7f80fdf89700 1 -- 172.21.15.49:0/2840819381 shutdown_connections 2024-03-26T17:53:10.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.598+0000 7f80fdf89700 1 --2- 172.21.15.49:0/2840819381 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f8102fa0 0x7f80f8103380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:10.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.598+0000 7f80fdf89700 1 -- 172.21.15.49:0/2840819381 >> 172.21.15.49:0/2840819381 conn(0x7f80f80fc770 msgr2=0x7f80f80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:10.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.598+0000 7f80fdf89700 1 -- 172.21.15.49:0/2840819381 shutdown_connections 2024-03-26T17:53:10.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.598+0000 7f80fdf89700 1 -- 172.21.15.49:0/2840819381 wait complete. 2024-03-26T17:53:10.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.598+0000 7f80fdf89700 1 Processor -- start 2024-03-26T17:53:10.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.598+0000 7f80fdf89700 1 -- start start 2024-03-26T17:53:10.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80fdf89700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f8102fa0 0x7f80f819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:10.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80fdf89700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f80f810c420 con 0x7f80f8102fa0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80f77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f8102fa0 0x7f80f819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80f77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f8102fa0 0x7f80f819d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39470/0 (socket says 172.21.15.49:39470) 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80f77fe700 1 -- 172.21.15.49:0/787570980 learned_addr learned my addr 172.21.15.49:0/787570980 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80f77fe700 1 -- 172.21.15.49:0/787570980 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f819db90 con 0x7f80f8102fa0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80f77fe700 1 --2- 172.21.15.49:0/787570980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f8102fa0 0x7f80f819d650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f80e8006010 tx=0x7f80e8005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80f4ff9700 1 -- 172.21.15.49:0/787570980 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e8026070 con 0x7f80f8102fa0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80f4ff9700 1 -- 172.21.15.49:0/787570980 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e800bc30 con 0x7f80f8102fa0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80fdf89700 1 -- 172.21.15.49:0/787570980 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f819de20 con 0x7f80f8102fa0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80f4ff9700 1 -- 172.21.15.49:0/787570980 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e8003ab0 con 0x7f80f8102fa0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.599+0000 7f80fdf89700 1 -- 172.21.15.49:0/787570980 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f8197720 con 0x7f80f8102fa0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.600+0000 7f80fdf89700 1 -- 172.21.15.49:0/787570980 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f806aa10 con 0x7f80f8102fa0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.600+0000 7f80f4ff9700 1 -- 172.21.15.49:0/787570980 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f80e8029070 con 0x7f80f8102fa0 2024-03-26T17:53:10.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.601+0000 7f80f4ff9700 1 --2- 172.21.15.49:0/787570980 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f80e0039220 0x7f80e003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:10.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.601+0000 7f80f4ff9700 1 -- 172.21.15.49:0/787570980 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f80e801e070 con 0x7f80f8102fa0 2024-03-26T17:53:10.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.601+0000 7f80f6ffd700 1 --2- 172.21.15.49:0/787570980 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f80e0039220 0x7f80e003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:10.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.601+0000 7f80f6ffd700 1 --2- 172.21.15.49:0/787570980 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f80e0039220 0x7f80e003b6e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f80ec006fd0 tx=0x7f80ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:10.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.606+0000 7f80f4ff9700 1 -- 172.21.15.49:0/787570980 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f80e8013070 con 0x7f80f8102fa0 2024-03-26T17:53:10.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:10.853+0000 7f80fdf89700 1 -- 172.21.15.49:0/787570980 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}) -- 0x7f80f8198c50 con 0x7f80e0039220 2024-03-26T17:53:12.010 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:12.010 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:12.010 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: Deploying cephadm binary to smithi171 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:12.011 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:12.012 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:14.010 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:13 smithi049 ceph-mon[32331]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:15.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.151+0000 7f80f4ff9700 1 -- 172.21.15.49:0/787570980 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f80f8198c50 con 0x7f80e0039220 2024-03-26T17:53:15.141 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi171' with addr '172.21.15.171' 2024-03-26T17:53:15.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.153+0000 7f80fdf89700 1 -- 172.21.15.49:0/787570980 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f80e0039220 msgr2=0x7f80e003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.153+0000 7f80fdf89700 1 --2- 172.21.15.49:0/787570980 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f80e0039220 0x7f80e003b6e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f80ec006fd0 tx=0x7f80ec006e40 comp rx=0 tx=0).stop 2024-03-26T17:53:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.153+0000 7f80fdf89700 1 -- 172.21.15.49:0/787570980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f8102fa0 msgr2=0x7f80f819d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.153+0000 7f80fdf89700 1 --2- 172.21.15.49:0/787570980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f8102fa0 0x7f80f819d650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f80e8006010 tx=0x7f80e8005050 comp rx=0 tx=0).stop 2024-03-26T17:53:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.153+0000 7f80fdf89700 1 -- 172.21.15.49:0/787570980 shutdown_connections 2024-03-26T17:53:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.153+0000 7f80fdf89700 1 --2- 172.21.15.49:0/787570980 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f80e0039220 0x7f80e003b6e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.153+0000 7f80fdf89700 1 --2- 172.21.15.49:0/787570980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f80f8102fa0 0x7f80f819d650 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.153+0000 7f80fdf89700 1 -- 172.21.15.49:0/787570980 >> 172.21.15.49:0/787570980 conn(0x7f80f80fc770 msgr2=0x7f80f8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.153+0000 7f80fdf89700 1 -- 172.21.15.49:0/787570980 shutdown_connections 2024-03-26T17:53:15.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:15.153+0000 7f80fdf89700 1 -- 172.21.15.49:0/787570980 wait complete. 2024-03-26T17:53:15.769 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T17:53:16.012 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:53:16.462 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:16 smithi049 ceph-mon[32331]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:16.462 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:16 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:16.462 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:16 smithi049 ceph-mon[32331]: Added host smithi171 2024-03-26T17:53:16.462 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:16 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:16.463 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:16 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:17.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.600+0000 7f721dfed700 1 -- 172.21.15.49:0/782937584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7218103010 msgr2=0x7f72181033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.600+0000 7f721dfed700 1 --2- 172.21.15.49:0/782937584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7218103010 0x7f72181033f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7208009910 tx=0x7f7208009c20 comp rx=0 tx=0).stop 2024-03-26T17:53:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.601+0000 7f721dfed700 1 -- 172.21.15.49:0/782937584 shutdown_connections 2024-03-26T17:53:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.601+0000 7f721dfed700 1 --2- 172.21.15.49:0/782937584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7218103010 0x7f72181033f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.601+0000 7f721dfed700 1 -- 172.21.15.49:0/782937584 >> 172.21.15.49:0/782937584 conn(0x7f72180fc780 msgr2=0x7f72180feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.601+0000 7f721dfed700 1 -- 172.21.15.49:0/782937584 shutdown_connections 2024-03-26T17:53:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.601+0000 7f721dfed700 1 -- 172.21.15.49:0/782937584 wait complete. 2024-03-26T17:53:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.601+0000 7f721dfed700 1 Processor -- start 2024-03-26T17:53:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.601+0000 7f721dfed700 1 -- start start 2024-03-26T17:53:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f721dfed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7218103010 0x7f7218100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f721dfed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f721810c430 con 0x7f7218103010 2024-03-26T17:53:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f72177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7218103010 0x7f7218100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f72177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7218103010 0x7f7218100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:39478/0 (socket says 172.21.15.49:39478) 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f72177fe700 1 -- 172.21.15.49:0/4092481604 learned_addr learned my addr 172.21.15.49:0/4092481604 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f72177fe700 1 -- 172.21.15.49:0/4092481604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7218102b60 con 0x7f7218103010 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f72177fe700 1 --2- 172.21.15.49:0/4092481604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7218103010 0x7f7218100c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7208006010 tx=0x7f7208005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f7214ff9700 1 -- 172.21.15.49:0/4092481604 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7208026070 con 0x7f7218103010 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f7214ff9700 1 -- 172.21.15.49:0/4092481604 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f720800bc30 con 0x7f7218103010 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f721dfed700 1 -- 172.21.15.49:0/4092481604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72181011b0 con 0x7f7218103010 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f7214ff9700 1 -- 172.21.15.49:0/4092481604 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7208003ab0 con 0x7f7218103010 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.602+0000 7f721dfed700 1 -- 172.21.15.49:0/4092481604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7218101670 con 0x7f7218103010 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.603+0000 7f7214ff9700 1 -- 172.21.15.49:0/4092481604 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f7208029070 con 0x7f7218103010 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.603+0000 7f721dfed700 1 -- 172.21.15.49:0/4092481604 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721806aa10 con 0x7f7218103010 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.603+0000 7f7214ff9700 1 --2- 172.21.15.49:0/4092481604 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7200039280 0x7f720003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.603+0000 7f7214ff9700 1 -- 172.21.15.49:0/4092481604 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f720801e070 con 0x7f7218103010 2024-03-26T17:53:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.603+0000 7f7216ffd700 1 --2- 172.21.15.49:0/4092481604 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7200039280 0x7f720003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.604+0000 7f7216ffd700 1 --2- 172.21.15.49:0/4092481604 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7200039280 0x7f720003b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f720c006fd0 tx=0x7f720c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:17.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.608+0000 7f7214ff9700 1 -- 172.21.15.49:0/4092481604 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7208013070 con 0x7f7218103010 2024-03-26T17:53:17.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.823+0000 7f721dfed700 1 -- 172.21.15.49:0/4092481604 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7218102210 con 0x7f7200039280 2024-03-26T17:53:17.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.824+0000 7f7214ff9700 1 -- 172.21.15.49:0/4092481604 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f7218102210 con 0x7f7200039280 2024-03-26T17:53:17.813 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:53:17.813 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2024-03-26T17:53:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.826+0000 7f721dfed700 1 -- 172.21.15.49:0/4092481604 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7200039280 msgr2=0x7f720003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.826+0000 7f721dfed700 1 --2- 172.21.15.49:0/4092481604 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7200039280 0x7f720003b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f720c006fd0 tx=0x7f720c006e40 comp rx=0 tx=0).stop 2024-03-26T17:53:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.826+0000 7f721dfed700 1 -- 172.21.15.49:0/4092481604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7218103010 msgr2=0x7f7218100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.826+0000 7f721dfed700 1 --2- 172.21.15.49:0/4092481604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7218103010 0x7f7218100c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7208006010 tx=0x7f7208005050 comp rx=0 tx=0).stop 2024-03-26T17:53:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.826+0000 7f721dfed700 1 -- 172.21.15.49:0/4092481604 shutdown_connections 2024-03-26T17:53:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.826+0000 7f721dfed700 1 --2- 172.21.15.49:0/4092481604 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7200039280 0x7f720003b740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.826+0000 7f721dfed700 1 --2- 172.21.15.49:0/4092481604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7218103010 0x7f7218100c70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.826+0000 7f721dfed700 1 -- 172.21.15.49:0/4092481604 >> 172.21.15.49:0/4092481604 conn(0x7f72180fc780 msgr2=0x7f7218103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.826+0000 7f721dfed700 1 -- 172.21.15.49:0/4092481604 shutdown_connections 2024-03-26T17:53:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:17.826+0000 7f721dfed700 1 -- 172.21.15.49:0/4092481604 wait complete. 2024-03-26T17:53:18.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:18 smithi049 ceph-mon[32331]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:18.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:18 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:18.399 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-26T17:53:18.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-26T17:53:18.647 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:53:19.495 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:19 smithi049 ceph-mon[32331]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:53:19.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.991+0000 7f83af87b700 1 -- 172.21.15.49:0/3701332584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a8100ee0 msgr2=0x7f83a81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:19.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.991+0000 7f83af87b700 1 --2- 172.21.15.49:0/3701332584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a8100ee0 0x7f83a81012c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009910 tx=0x7f83a4009c20 comp rx=0 tx=0).stop 2024-03-26T17:53:19.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.991+0000 7f83af87b700 1 -- 172.21.15.49:0/3701332584 shutdown_connections 2024-03-26T17:53:19.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.991+0000 7f83af87b700 1 --2- 172.21.15.49:0/3701332584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a8100ee0 0x7f83a81012c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.991+0000 7f83af87b700 1 -- 172.21.15.49:0/3701332584 >> 172.21.15.49:0/3701332584 conn(0x7f83a80778f0 msgr2=0x7f83a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:19.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.991+0000 7f83af87b700 1 -- 172.21.15.49:0/3701332584 shutdown_connections 2024-03-26T17:53:19.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.991+0000 7f83af87b700 1 -- 172.21.15.49:0/3701332584 wait complete. 2024-03-26T17:53:19.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.992+0000 7f83af87b700 1 Processor -- start 2024-03-26T17:53:19.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.992+0000 7f83af87b700 1 -- start start 2024-03-26T17:53:19.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.992+0000 7f83af87b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a8100ee0 0x7f83a819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.992+0000 7f83af87b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f83a810b2b0 con 0x7f83a8100ee0 2024-03-26T17:53:19.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.992+0000 7f83ad617700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a8100ee0 0x7f83a819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.992+0000 7f83ad617700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a8100ee0 0x7f83a819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53956/0 (socket says 172.21.15.49:53956) 2024-03-26T17:53:19.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.992+0000 7f83ad617700 1 -- 172.21.15.49:0/3520697035 learned_addr learned my addr 172.21.15.49:0/3520697035 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:53:19.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.992+0000 7f83ad617700 1 -- 172.21.15.49:0/3520697035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a819db30 con 0x7f83a8100ee0 2024-03-26T17:53:19.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.993+0000 7f83ad617700 1 --2- 172.21.15.49:0/3520697035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a8100ee0 0x7f83a819d5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f83a4005850 tx=0x7f83a4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:19.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.993+0000 7f839a7fc700 1 -- 172.21.15.49:0/3520697035 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83a4026070 con 0x7f83a8100ee0 2024-03-26T17:53:19.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.993+0000 7f839a7fc700 1 -- 172.21.15.49:0/3520697035 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83a400bbd0 con 0x7f83a8100ee0 2024-03-26T17:53:19.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.993+0000 7f83af87b700 1 -- 172.21.15.49:0/3520697035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a819de20 con 0x7f83a8100ee0 2024-03-26T17:53:19.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.993+0000 7f839a7fc700 1 -- 172.21.15.49:0/3520697035 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83a4003970 con 0x7f83a8100ee0 2024-03-26T17:53:19.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.993+0000 7f83af87b700 1 -- 172.21.15.49:0/3520697035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a8197890 con 0x7f83a8100ee0 2024-03-26T17:53:19.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.994+0000 7f839a7fc700 1 -- 172.21.15.49:0/3520697035 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f83a4029070 con 0x7f83a8100ee0 2024-03-26T17:53:19.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.994+0000 7f83af87b700 1 -- 172.21.15.49:0/3520697035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a81023c0 con 0x7f83a8100ee0 2024-03-26T17:53:19.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.994+0000 7f839a7fc700 1 --2- 172.21.15.49:0/3520697035 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8394039280 0x7f839403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.994+0000 7f839a7fc700 1 -- 172.21.15.49:0/3520697035 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f83a401e070 con 0x7f83a8100ee0 2024-03-26T17:53:19.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.994+0000 7f83ace16700 1 --2- 172.21.15.49:0/3520697035 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8394039280 0x7f839403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.994+0000 7f83ace16700 1 --2- 172.21.15.49:0/3520697035 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8394039280 0x7f839403b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f839c006fd0 tx=0x7f839c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:19.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:19.999+0000 7f839a7fc700 1 -- 172.21.15.49:0/3520697035 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f83a4013070 con 0x7f83a8100ee0 2024-03-26T17:53:20.198 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:20.209+0000 7f83af87b700 1 -- 172.21.15.49:0/3520697035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f83a8198bf0 con 0x7f83a8100ee0 2024-03-26T17:53:20.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:20 smithi049 ceph-mon[32331]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:21.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.179+0000 7f839a7fc700 1 -- 172.21.15.49:0/3520697035 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f83a4030090 con 0x7f83a8100ee0 2024-03-26T17:53:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.182+0000 7f83af87b700 1 -- 172.21.15.49:0/3520697035 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8394039280 msgr2=0x7f839403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.182+0000 7f83af87b700 1 --2- 172.21.15.49:0/3520697035 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8394039280 0x7f839403b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f839c006fd0 tx=0x7f839c006e40 comp rx=0 tx=0).stop 2024-03-26T17:53:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.182+0000 7f83af87b700 1 -- 172.21.15.49:0/3520697035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a8100ee0 msgr2=0x7f83a819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.182+0000 7f83af87b700 1 --2- 172.21.15.49:0/3520697035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a8100ee0 0x7f83a819d5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f83a4005850 tx=0x7f83a4005030 comp rx=0 tx=0).stop 2024-03-26T17:53:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.182+0000 7f83af87b700 1 -- 172.21.15.49:0/3520697035 shutdown_connections 2024-03-26T17:53:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.182+0000 7f83af87b700 1 --2- 172.21.15.49:0/3520697035 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8394039280 0x7f839403b740 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.182+0000 7f83af87b700 1 --2- 172.21.15.49:0/3520697035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83a8100ee0 0x7f83a819d5f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:21.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.182+0000 7f83af87b700 1 -- 172.21.15.49:0/3520697035 >> 172.21.15.49:0/3520697035 conn(0x7f83a80778f0 msgr2=0x7f83a80697a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.182+0000 7f83af87b700 1 -- 172.21.15.49:0/3520697035 shutdown_connections 2024-03-26T17:53:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:21.183+0000 7f83af87b700 1 -- 172.21.15.49:0/3520697035 wait complete. 2024-03-26T17:53:21.173 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2024-03-26T17:53:21.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:21 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3520697035' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T17:53:21.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:21.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:21.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:21.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:21.485 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:53:21.486 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:21.486 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:21.799 INFO:tasks.cephadm:Adding mon.a on smithi049 2024-03-26T17:53:21.799 INFO:tasks.cephadm:Adding mon.b on smithi115 2024-03-26T17:53:21.799 INFO:tasks.cephadm:Adding mon.c on smithi171 2024-03-26T17:53:21.800 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi049:172.21.15.49=a;smithi115:172.21.15.115=b;smithi171:172.21.15.171=c' 2024-03-26T17:53:22.029 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:22.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:22 smithi049 ceph-mon[32331]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:22.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:22 smithi049 ceph-mon[32331]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-26T17:53:22.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:22 smithi049 ceph-mon[32331]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:22.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:22 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3520697035' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T17:53:22.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:22 smithi049 ceph-mon[32331]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:53:22.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:22 smithi049 ceph-mon[32331]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:22.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:22.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:22.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:23.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:23 smithi049 ceph-mon[32331]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:53:23.542 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.549+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2331489718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7e41064b0 msgr2=0x7fd7e4106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:23.542 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.549+0000 7fd7ec7ab700 1 --2- 172.21.15.171:0/2331489718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7e41064b0 0x7fd7e4106890 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00c210 tx=0x7fd7dc00c520 comp rx=0 tx=0).stop 2024-03-26T17:53:23.542 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.549+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2331489718 shutdown_connections 2024-03-26T17:53:23.542 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.549+0000 7fd7ec7ab700 1 --2- 172.21.15.171:0/2331489718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7e41064b0 0x7fd7e4106890 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:23.542 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.549+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2331489718 >> 172.21.15.171:0/2331489718 conn(0x7fd7e4077360 msgr2=0x7fd7e4077770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:23.542 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.549+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2331489718 shutdown_connections 2024-03-26T17:53:23.542 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.549+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2331489718 wait complete. 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.549+0000 7fd7ec7ab700 1 Processor -- start 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.549+0000 7fd7ec7ab700 1 -- start start 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.550+0000 7fd7ec7ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7e41064b0 0x7fd7e41956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.550+0000 7fd7ec7ab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e410a1e0 con 0x7fd7e41064b0 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.550+0000 7fd7ea547700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7e41064b0 0x7fd7e41956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.550+0000 7fd7ea547700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7e41064b0 0x7fd7e41956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.171:54194/0 (socket says 172.21.15.171:54194) 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.550+0000 7fd7ea547700 1 -- 172.21.15.171:0/2785245906 learned_addr learned my addr 172.21.15.171:0/2785245906 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.550+0000 7fd7ea547700 1 -- 172.21.15.171:0/2785245906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e4195c00 con 0x7fd7e41064b0 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.550+0000 7fd7ea547700 1 --2- 172.21.15.171:0/2785245906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7e41064b0 0x7fd7e41956c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc016040 tx=0x7fd7dc008d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.551+0000 7fd7e37fe700 1 -- 172.21.15.171:0/2785245906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7dc020020 con 0x7fd7e41064b0 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.551+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2785245906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e4195e90 con 0x7fd7e41064b0 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.551+0000 7fd7e37fe700 1 -- 172.21.15.171:0/2785245906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7dc003d50 con 0x7fd7e41064b0 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.551+0000 7fd7e37fe700 1 -- 172.21.15.171:0/2785245906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7dc018040 con 0x7fd7e41064b0 2024-03-26T17:53:23.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.551+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2785245906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e4199f90 con 0x7fd7e41064b0 2024-03-26T17:53:23.544 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.551+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2785245906 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e4101b60 con 0x7fd7e41064b0 2024-03-26T17:53:23.544 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.552+0000 7fd7e37fe700 1 -- 172.21.15.171:0/2785245906 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fd7dc023020 con 0x7fd7e41064b0 2024-03-26T17:53:23.544 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.552+0000 7fd7e37fe700 1 --2- 172.21.15.171:0/2785245906 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7cc039220 0x7fd7cc03b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:23.544 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.552+0000 7fd7e37fe700 1 -- 172.21.15.171:0/2785245906 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd7dc010050 con 0x7fd7e41064b0 2024-03-26T17:53:23.544 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.552+0000 7fd7e9d46700 1 --2- 172.21.15.171:0/2785245906 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7cc039220 0x7fd7cc03b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:23.544 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.553+0000 7fd7e9d46700 1 --2- 172.21.15.171:0/2785245906 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7cc039220 0x7fd7cc03b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8006fd0 tx=0x7fd7d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:23.547 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.557+0000 7fd7e37fe700 1 -- 172.21.15.171:0/2785245906 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd7dc02a020 con 0x7fd7e41064b0 2024-03-26T17:53:23.757 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.767+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2785245906 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi115:172.21.15.115=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}) -- 0x7fd7e4196a20 con 0x7fd7cc039220 2024-03-26T17:53:23.763 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.774+0000 7fd7e37fe700 1 -- 172.21.15.171:0/2785245906 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd7e4196a20 con 0x7fd7cc039220 2024-03-26T17:53:23.764 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mon update... 2024-03-26T17:53:23.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.776+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2785245906 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7cc039220 msgr2=0x7fd7cc03b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:23.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.776+0000 7fd7ec7ab700 1 --2- 172.21.15.171:0/2785245906 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7cc039220 0x7fd7cc03b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8006fd0 tx=0x7fd7d8006e40 comp rx=0 tx=0).stop 2024-03-26T17:53:23.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.776+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2785245906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7e41064b0 msgr2=0x7fd7e41956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:23.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.776+0000 7fd7ec7ab700 1 --2- 172.21.15.171:0/2785245906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7e41064b0 0x7fd7e41956c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc016040 tx=0x7fd7dc008d60 comp rx=0 tx=0).stop 2024-03-26T17:53:23.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.776+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2785245906 shutdown_connections 2024-03-26T17:53:23.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.776+0000 7fd7ec7ab700 1 --2- 172.21.15.171:0/2785245906 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7cc039220 0x7fd7cc03b6e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:23.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.776+0000 7fd7ec7ab700 1 --2- 172.21.15.171:0/2785245906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7e41064b0 0x7fd7e41956c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:23.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.776+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2785245906 >> 172.21.15.171:0/2785245906 conn(0x7fd7e4077360 msgr2=0x7fd7e40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:23.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.776+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2785245906 shutdown_connections 2024-03-26T17:53:23.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:23.776+0000 7fd7ec7ab700 1 -- 172.21.15.171:0/2785245906 wait complete. 2024-03-26T17:53:24.255 DEBUG:teuthology.orchestra.run.smithi115:mon.b> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.b.service 2024-03-26T17:53:24.258 DEBUG:teuthology.orchestra.run.smithi171:mon.c> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.c.service 2024-03-26T17:53:24.260 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T17:53:24.260 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T17:53:24.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:24 smithi049 ceph-mon[32331]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:24.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:24.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:24.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:24.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:24.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:24.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:24.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:25.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:25 smithi049 ceph-mon[32331]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi115:172.21.15.115=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:53:25.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:25 smithi049 ceph-mon[32331]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi115:172.21.15.115=b;smithi171:172.21.15.171=c;count:3 2024-03-26T17:53:25.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:25 smithi049 ceph-mon[32331]: Deploying daemon mon.c on smithi171 2024-03-26T17:53:26.250 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:53:26.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:26 smithi049 ceph-mon[32331]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:27.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:27 smithi171 podman[35983]: 2024-03-26 17:53:27.894889999 +0000 UTC m=+0.359780117 container create 4f0304e95a283bf5066bea295d4391dcf0c7e83dafb1a699c8bd2286ef094558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-26T17:53:28.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.196+0000 7fd8862dc700 1 -- 172.21.15.171:0/325491523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd880102110 msgr2=0x7fd8801024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:28.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.196+0000 7fd8862dc700 1 --2- 172.21.15.171:0/325491523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd880102110 0x7fd8801024f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd870009910 tx=0x7fd870009c20 comp rx=0 tx=0).stop 2024-03-26T17:53:28.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.196+0000 7fd8862dc700 1 -- 172.21.15.171:0/325491523 shutdown_connections 2024-03-26T17:53:28.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.196+0000 7fd8862dc700 1 --2- 172.21.15.171:0/325491523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd880102110 0x7fd8801024f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:28.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.196+0000 7fd8862dc700 1 -- 172.21.15.171:0/325491523 >> 172.21.15.171:0/325491523 conn(0x7fd8800fd9a0 msgr2=0x7fd8800ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:28.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.196+0000 7fd8862dc700 1 -- 172.21.15.171:0/325491523 shutdown_connections 2024-03-26T17:53:28.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.196+0000 7fd8862dc700 1 -- 172.21.15.171:0/325491523 wait complete. 2024-03-26T17:53:28.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.197+0000 7fd8862dc700 1 Processor -- start 2024-03-26T17:53:28.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.197+0000 7fd8862dc700 1 -- start start 2024-03-26T17:53:28.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.197+0000 7fd8862dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd880102110 0x7fd88019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.197+0000 7fd8862dc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd88010c4e0 con 0x7fd880102110 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.197+0000 7fd87ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd880102110 0x7fd88019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.197+0000 7fd87ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd880102110 0x7fd88019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.171:54208/0 (socket says 172.21.15.171:54208) 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.197+0000 7fd87ffff700 1 -- 172.21.15.171:0/1253734138 learned_addr learned my addr 172.21.15.171:0/1253734138 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.198+0000 7fd87ffff700 1 -- 172.21.15.171:0/1253734138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88019dbe0 con 0x7fd880102110 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.198+0000 7fd87ffff700 1 --2- 172.21.15.171:0/1253734138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd880102110 0x7fd88019d6a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd870005850 tx=0x7fd870005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.198+0000 7fd87d7fa700 1 -- 172.21.15.171:0/1253734138 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd870026070 con 0x7fd880102110 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.198+0000 7fd8862dc700 1 -- 172.21.15.171:0/1253734138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88019ded0 con 0x7fd880102110 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.198+0000 7fd87d7fa700 1 -- 172.21.15.171:0/1253734138 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87000bbd0 con 0x7fd880102110 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.198+0000 7fd87d7fa700 1 -- 172.21.15.171:0/1253734138 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd870003970 con 0x7fd880102110 2024-03-26T17:53:28.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.198+0000 7fd8862dc700 1 -- 172.21.15.171:0/1253734138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd880197820 con 0x7fd880102110 2024-03-26T17:53:28.190 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.199+0000 7fd8862dc700 1 -- 172.21.15.171:0/1253734138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8801035f0 con 0x7fd880102110 2024-03-26T17:53:28.190 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.199+0000 7fd87d7fa700 1 -- 172.21.15.171:0/1253734138 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7fd870029070 con 0x7fd880102110 2024-03-26T17:53:28.190 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.199+0000 7fd87d7fa700 1 --2- 172.21.15.171:0/1253734138 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd868039280 0x7fd86803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:28.191 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.199+0000 7fd87d7fa700 1 -- 172.21.15.171:0/1253734138 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd87001e070 con 0x7fd880102110 2024-03-26T17:53:28.191 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.200+0000 7fd87f7fe700 1 --2- 172.21.15.171:0/1253734138 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd868039280 0x7fd86803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:28.191 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.200+0000 7fd87f7fe700 1 --2- 172.21.15.171:0/1253734138 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd868039280 0x7fd86803b740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:28.194 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.205+0000 7fd87d7fa700 1 -- 172.21.15.171:0/1253734138 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd870013070 con 0x7fd880102110 2024-03-26T17:53:28.312 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.322+0000 7fd87d7fa700 1 -- 172.21.15.171:0/1253734138 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd87000e3e0 con 0x7fd880102110 2024-03-26T17:53:28.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:28.488+0000 7fd8862dc700 1 -- 172.21.15.171:0/1253734138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd880198d50 con 0x7fd880102110 2024-03-26T17:53:28.478 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 podman[35983]: 2024-03-26 17:53:28.265069649 +0000 UTC m=+0.729959763 container init 4f0304e95a283bf5066bea295d4391dcf0c7e83dafb1a699c8bd2286ef094558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T17:53:28.478 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 podman[35983]: 2024-03-26 17:53:28.269167062 +0000 UTC m=+0.734057166 container start 4f0304e95a283bf5066bea295d4391dcf0c7e83dafb1a699c8bd2286ef094558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T17:53:28.478 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev), process ceph-mon, pid 2 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pidfile_write: ignore empty --pid-file 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: load: jerasure load: lrc 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: RocksDB version: 7.9.2 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Git sha 0 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Compile date 2024-03-20 16:36:06 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: DB SUMMARY 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: DB Session ID: 2GLFLS6KHUH6CJKUK1XU 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: CURRENT file: CURRENT 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T17:53:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.error_if_exists: 0 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.create_if_missing: 0 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.paranoid_checks: 1 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.env: 0x556819048c60 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.info_log: 0x55681ac9dae0 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T17:53:28.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.statistics: (nil) 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.use_fsync: 0 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_log_file_size: 0 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.allow_fallocate: 1 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.use_direct_reads: 0 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T17:53:28.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.db_log_dir: 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.wal_dir: 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.write_buffer_manager: 0x55681ac701e0 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T17:53:28.482 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.unordered_write: 0 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.row_cache: None 2024-03-26T17:53:28.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.wal_filter: None 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.two_write_queues: 0 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.wal_compression: 0 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.atomic_flush: 0 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.log_readahead_size: 0 2024-03-26T17:53:28.485 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_background_jobs: 2 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_background_compactions: -1 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_subcompactions: 1 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T17:53:28.486 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_open_files: -1 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_background_flushes: -1 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Compression algorithms supported: 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: kZSTD supported: 0 2024-03-26T17:53:28.487 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: kXpressCompression supported: 0 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: kZlibCompression supported: 1 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: kSnappyCompression supported: 1 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: kLZ4Compression supported: 1 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: kBZip2Compression supported: 0 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.merge_operator: 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_filter: None 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T17:53:28.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55681ac9dbe0) 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks: 1 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_top_level_index_and_filter: 1 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_type: 0 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_index_type: 0 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_shortening: 1 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: checksum: 4 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: no_block_cache: 0 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache: 0x55681aca3090 2024-03-26T17:53:28.489 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_name: BinnedLRUCache 2024-03-26T17:53:28.490 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_options: 2024-03-26T17:53:28.490 INFO:journalctl@ceph.mon.c.smithi171.stdout: capacity : 536870912 2024-03-26T17:53:28.490 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_shard_bits : 4 2024-03-26T17:53:28.490 INFO:journalctl@ceph.mon.c.smithi171.stdout: strict_capacity_limit : 0 2024-03-26T17:53:28.490 INFO:journalctl@ceph.mon.c.smithi171.stdout: high_pri_pool_ratio: 0.000 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_compressed: (nil) 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: persistent_cache: (nil) 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size: 4096 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size_deviation: 10 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_restart_interval: 16 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_block_restart_interval: 1 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: metadata_block_size: 4096 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: partition_filters: 0 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: use_delta_encoding: 1 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: filter_policy: bloomfilter 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: whole_key_filtering: 1 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: verify_compression: 0 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: read_amp_bytes_per_bit: 0 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: format_version: 5 2024-03-26T17:53:28.491 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_index_compression: 1 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_align: 0 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout: max_auto_readahead_size: 262144 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout: prepopulate_block_cache: 0 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout: initial_auto_readahead_size: 8192 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compression: NoCompression 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.num_levels: 7 2024-03-26T17:53:28.492 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T17:53:28.493 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T17:53:28.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T17:53:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.table_properties_collectors: 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.inplace_update_support: 0 2024-03-26T17:53:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.bloom_locality: 0 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.max_successive_merges: 0 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.ttl: 2592000 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T17:53:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.enable_blob_files: false 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.min_blob_size: 0 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f641a24b-ff16-4ab4-9afa-8f664767ec39 2024-03-26T17:53:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711475608299135, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711475608299578, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711475608, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f641a24b-ff16-4ab4-9afa-8f664767ec39", "db_session_id": "2GLFLS6KHUH6CJKUK1XU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711475608299658, "job": 1, "event": "recovery_finished"} 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55681ad96000 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: DB pointer 0x55681ad80000 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** DB Stats ** 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T17:53:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-03-26T17:53:28.500 INFO:journalctl@ceph.mon.c.smithi171.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T17:53:28.501 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache BinnedLRUCache@0x55681aca3090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: using public_addr v2:172.21.15.171:0/0 -> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: starting mon.c rank -1 at public addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] at bind addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(???) e0 preinit fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).mds e1 new map 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout: e1 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T17:53:28.502 INFO:journalctl@ceph.mon.c.smithi171.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout: legacy client fscid: -1 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout: No filesystems configured 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:53:28.503 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:28.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Deploying cephadm binary to smithi171 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Added host smithi171 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3520697035' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:28.506 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3520697035' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:28.507 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:28.508 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:28.508 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:28.508 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi115:172.21.15.115=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:53:28.508 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi115:172.21.15.115=b;smithi171:172.21.15.171=c;count:3 2024-03-26T17:53:28.508 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: Deploying daemon mon.c on smithi171 2024-03-26T17:53:28.508 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:28.508 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:28.508 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 ceph-mon[36033]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T17:53:28.508 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 bash[35983]: 4f0304e95a283bf5066bea295d4391dcf0c7e83dafb1a699c8bd2286ef094558 2024-03-26T17:53:28.508 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:28 smithi171 systemd[1]: Started Ceph mon.c for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:53:28.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:28 smithi049 ceph-mon[32331]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:33.359 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.369+0000 7fd87d7fa700 1 -- 172.21.15.171:0/1253734138 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fd870030020 con 0x7fd880102110 2024-03-26T17:53:33.359 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:53:33.359 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","modified":"2024-03-26T17:53:28.322709Z","created":"2024-03-26T17:51:39.354207Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T17:53:33.362 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.372+0000 7fd8862dc700 1 -- 172.21.15.171:0/1253734138 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd868039280 msgr2=0x7fd86803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:33.362 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.372+0000 7fd8862dc700 1 --2- 172.21.15.171:0/1253734138 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd868039280 0x7fd86803b740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874006e40 comp rx=0 tx=0).stop 2024-03-26T17:53:33.362 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.372+0000 7fd8862dc700 1 -- 172.21.15.171:0/1253734138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd880102110 msgr2=0x7fd88019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:33.362 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.372+0000 7fd8862dc700 1 --2- 172.21.15.171:0/1253734138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd880102110 0x7fd88019d6a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd870005850 tx=0x7fd870005030 comp rx=0 tx=0).stop 2024-03-26T17:53:33.362 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.372+0000 7fd8862dc700 1 -- 172.21.15.171:0/1253734138 shutdown_connections 2024-03-26T17:53:33.362 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.372+0000 7fd8862dc700 1 --2- 172.21.15.171:0/1253734138 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd868039280 0x7fd86803b740 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:33.362 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.372+0000 7fd8862dc700 1 --2- 172.21.15.171:0/1253734138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd880102110 0x7fd88019d6a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:33.362 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.372+0000 7fd8862dc700 1 -- 172.21.15.171:0/1253734138 >> 172.21.15.171:0/1253734138 conn(0x7fd8800fd9a0 msgr2=0x7fd880103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:33.362 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.372+0000 7fd8862dc700 1 -- 172.21.15.171:0/1253734138 shutdown_connections 2024-03-26T17:53:33.362 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:33.372+0000 7fd8862dc700 1 -- 172.21.15.171:0/1253734138 wait complete. 2024-03-26T17:53:33.363 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-03-26T17:53:33.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:53:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: mon.a calling monitor election 2024-03-26T17:53:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='client.? 172.21.15.171:0/1253734138' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T17:53:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: mon.c calling monitor election 2024-03-26T17:53:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: monmap epoch 2 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: last_changed 2024-03-26T17:53:28.322709+0000 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: created 2024-03-26T17:51:39.354207+0000 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: min_mon_release 19 (squid) 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: election_strategy: 1 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: fsmap 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: mgrmap e13: x(active, since 50s) 2024-03-26T17:53:33.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: overall HEALTH_OK 2024-03-26T17:53:33.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:33.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:33.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:33.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:33.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: mon.a calling monitor election 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='client.? 172.21.15.171:0/1253734138' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: mon.c calling monitor election 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: monmap epoch 2 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: last_changed 2024-03-26T17:53:28.322709+0000 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: created 2024-03-26T17:51:39.354207+0000 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: min_mon_release 19 (squid) 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: election_strategy: 1 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: fsmap 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: mgrmap e13: x(active, since 50s) 2024-03-26T17:53:33.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: overall HEALTH_OK 2024-03-26T17:53:33.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:33.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:33.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:33.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:33.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:34.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:34 smithi171 ceph-mon[36033]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:34.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:34 smithi171 ceph-mon[36033]: Deploying daemon mon.b on smithi115 2024-03-26T17:53:34.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:34 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:34.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:34 smithi049 ceph-mon[32331]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:34.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:34 smithi049 ceph-mon[32331]: Deploying daemon mon.b on smithi115 2024-03-26T17:53:34.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:34 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:34.988 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T17:53:34.988 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T17:53:35.236 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:53:36.642 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.650+0000 7f3641324700 1 -- 172.21.15.171:0/2634096942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f363c102100 msgr2=0x7f363c1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:36.642 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.650+0000 7f3641324700 1 --2- 172.21.15.171:0/2634096942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f363c102100 0x7f363c1024e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f363400b0a0 tx=0x7f363400b3b0 comp rx=0 tx=0).stop 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.650+0000 7f3641324700 1 -- 172.21.15.171:0/2634096942 shutdown_connections 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.650+0000 7f3641324700 1 --2- 172.21.15.171:0/2634096942 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f363c102100 0x7f363c1024e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.650+0000 7f3641324700 1 -- 172.21.15.171:0/2634096942 >> 172.21.15.171:0/2634096942 conn(0x7f363c0fd970 msgr2=0x7f363c0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.650+0000 7f3641324700 1 -- 172.21.15.171:0/2634096942 shutdown_connections 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.650+0000 7f3641324700 1 -- 172.21.15.171:0/2634096942 wait complete. 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.650+0000 7f3641324700 1 Processor -- start 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f3641324700 1 -- start start 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f3641324700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f363c102100 0x7f363c0746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f3641324700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f363c074bf0 0x7f363c075070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f3641324700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10c4d0 con 0x7f363c102100 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f3641324700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10c350 con 0x7f363c074bf0 2024-03-26T17:53:36.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363a7fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f363c074bf0 0x7f363c075070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363a7fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f363c074bf0 0x7f363c075070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:35750/0 (socket says 172.21.15.171:35750) 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363a7fc700 1 -- 172.21.15.171:0/2140630031 learned_addr learned my addr 172.21.15.171:0/2140630031 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363affd700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f363c102100 0x7f363c0746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363a7fc700 1 -- 172.21.15.171:0/2140630031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f363c074bf0 msgr2=0x7f363c075070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363a7fc700 1 -- 172.21.15.171:0/2140630031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f363c074bf0 msgr2=0x7f363c075070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363a7fc700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f363c074bf0 0x7f363c075070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363a7fc700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f363c074bf0 0x7f363c075070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363affd700 1 -- 172.21.15.171:0/2140630031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f363c074bf0 msgr2=0x7f363c075070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363affd700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f363c074bf0 0x7f363c075070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.651+0000 7f363affd700 1 -- 172.21.15.171:0/2140630031 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c0765d0 con 0x7f363c102100 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.652+0000 7f363affd700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f363c102100 0x7f363c0746b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36340062a0 tx=0x7f3634007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.652+0000 7f3623fff700 1 -- 172.21.15.171:0/2140630031 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3634007bd0 con 0x7f363c102100 2024-03-26T17:53:36.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.652+0000 7f3641324700 1 -- 172.21.15.171:0/2140630031 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c1a3d80 con 0x7f363c102100 2024-03-26T17:53:36.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.652+0000 7f3641324700 1 -- 172.21.15.171:0/2140630031 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1a4260 con 0x7f363c102100 2024-03-26T17:53:36.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.652+0000 7f3623fff700 1 -- 172.21.15.171:0/2140630031 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3634009ca0 con 0x7f363c102100 2024-03-26T17:53:36.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.652+0000 7f3623fff700 1 -- 172.21.15.171:0/2140630031 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3634012750 con 0x7f363c102100 2024-03-26T17:53:36.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.653+0000 7f3641324700 1 -- 172.21.15.171:0/2140630031 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3628005330 con 0x7f363c102100 2024-03-26T17:53:36.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.653+0000 7f3623fff700 1 -- 172.21.15.171:0/2140630031 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f3634024020 con 0x7f363c102100 2024-03-26T17:53:36.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.653+0000 7f3623fff700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3624038fc0 0x7f362403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:36.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.653+0000 7f3623fff700 1 -- 172.21.15.171:0/2140630031 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3634022020 con 0x7f363c102100 2024-03-26T17:53:36.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.653+0000 7f363a7fc700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3624038fc0 0x7f362403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:36.645 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.654+0000 7f363a7fc700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3624038fc0 0x7f362403b480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3630009180 tx=0x7f3630008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:36.648 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.658+0000 7f3623fff700 1 -- 172.21.15.171:0/2140630031 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f36340192f0 con 0x7f363c102100 2024-03-26T17:53:36.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:36 smithi049 ceph-mon[32331]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:36.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:36 smithi171 ceph-mon[36033]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:36.921 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.932+0000 7f3641324700 1 -- 172.21.15.171:0/2140630031 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3628005620 con 0x7f363c102100 2024-03-26T17:53:36.922 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.933+0000 7f3623fff700 1 -- 172.21.15.171:0/2140630031 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f36340259b0 con 0x7f363c102100 2024-03-26T17:53:36.923 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:53:36.923 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","modified":"2024-03-26T17:53:28.322709Z","created":"2024-03-26T17:51:39.354207Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T17:53:36.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 -- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3624038fc0 msgr2=0x7f362403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:36.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3624038fc0 0x7f362403b480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3630009180 tx=0x7f3630008040 comp rx=0 tx=0).stop 2024-03-26T17:53:36.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 -- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f363c102100 msgr2=0x7f363c0746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:36.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f363c102100 0x7f363c0746b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36340062a0 tx=0x7f3634007930 comp rx=0 tx=0).stop 2024-03-26T17:53:36.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 -- 172.21.15.171:0/2140630031 shutdown_connections 2024-03-26T17:53:36.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3624038fc0 0x7f362403b480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:36.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f363c102100 0x7f363c0746b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:36.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 --2- 172.21.15.171:0/2140630031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f363c074bf0 0x7f363c075070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:36.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 -- 172.21.15.171:0/2140630031 >> 172.21.15.171:0/2140630031 conn(0x7f363c0fd970 msgr2=0x7f363c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:36.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 -- 172.21.15.171:0/2140630031 shutdown_connections 2024-03-26T17:53:36.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:36.935+0000 7f3641324700 1 -- 172.21.15.171:0/2140630031 wait complete. 2024-03-26T17:53:36.927 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-03-26T17:53:37.053 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-26T17:53:37.054 INFO:journalctl@ceph.mon.b.smithi115.stdout: e1 2024-03-26T17:53:37.054 INFO:journalctl@ceph.mon.b.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T17:53:37.054 INFO:journalctl@ceph.mon.b.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T17:53:37.054 INFO:journalctl@ceph.mon.b.smithi115.stdout: legacy client fscid: -1 2024-03-26T17:53:37.054 INFO:journalctl@ceph.mon.b.smithi115.stdout: 2024-03-26T17:53:37.054 INFO:journalctl@ceph.mon.b.smithi115.stdout: No filesystems configured 2024-03-26T17:53:37.054 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-26T17:53:37.054 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-26T17:53:37.054 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-26T17:53:37.054 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.055 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Deploying cephadm binary to smithi171 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.056 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.057 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Added host smithi171 2024-03-26T17:53:37.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:37.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:53:37.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3520697035' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T17:53:37.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3520697035' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi115:172.21.15.115=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi115:172.21.15.115=b;smithi171:172.21.15.171=c;count:3 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Deploying daemon mon.c on smithi171 2024-03-26T17:53:37.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:53:37.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.a calling monitor election 2024-03-26T17:53:37.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:37.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='client.? 172.21.15.171:0/1253734138' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T17:53:37.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:37.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:37.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.c calling monitor election 2024-03-26T17:53:37.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:37.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:37.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:37.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T17:53:37.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: monmap epoch 2 2024-03-26T17:53:37.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:53:37.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: last_changed 2024-03-26T17:53:28.322709+0000 2024-03-26T17:53:37.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: created 2024-03-26T17:51:39.354207+0000 2024-03-26T17:53:37.499 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: min_mon_release 19 (squid) 2024-03-26T17:53:37.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: election_strategy: 1 2024-03-26T17:53:37.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-26T17:53:37.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-03-26T17:53:37.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: fsmap 2024-03-26T17:53:37.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:53:37.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mgrmap e13: x(active, since 50s) 2024-03-26T17:53:37.500 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: overall HEALTH_OK 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: Deploying daemon mon.b on smithi115 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 ceph-mon[36174]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 bash[36155]: e80310a4705ba6b42f0b5770adf200a32a03e4b8948209e5637a3c8777308df7 2024-03-26T17:53:37.501 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:37 smithi115 systemd[1]: Started Ceph mon.b for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:53:38.378 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T17:53:38.378 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T17:53:38.622 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:53:42.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:42.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:53:42.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: mon.a calling monitor election 2024-03-26T17:53:42.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: mon.c calling monitor election 2024-03-26T17:53:42.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:42.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: mon.b calling monitor election 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: monmap epoch 3 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: last_changed 2024-03-26T17:53:37.069267+0000 2024-03-26T17:53:42.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: created 2024-03-26T17:51:39.354207+0000 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: min_mon_release 19 (squid) 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: election_strategy: 1 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: fsmap 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: mgrmap e13: x(active, since 59s) 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: overall HEALTH_OK 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.497 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.498 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:42.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:53:42.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: mon.a calling monitor election 2024-03-26T17:53:42.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: mon.c calling monitor election 2024-03-26T17:53:42.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:42.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: mon.b calling monitor election 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: monmap epoch 3 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: last_changed 2024-03-26T17:53:37.069267+0000 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: created 2024-03-26T17:51:39.354207+0000 2024-03-26T17:53:42.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: min_mon_release 19 (squid) 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: election_strategy: 1 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: fsmap 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: mgrmap e13: x(active, since 59s) 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: overall HEALTH_OK 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.512 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:42.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: mon.a calling monitor election 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: mon.c calling monitor election 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: mon.b calling monitor election 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: monmap epoch 3 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: last_changed 2024-03-26T17:53:37.069267+0000 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: created 2024-03-26T17:51:39.354207+0000 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: min_mon_release 19 (squid) 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: election_strategy: 1 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: 0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: 2: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: fsmap 2024-03-26T17:53:42.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:53:42.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: mgrmap e13: x(active, since 59s) 2024-03-26T17:53:42.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: overall HEALTH_OK 2024-03-26T17:53:42.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:42.952 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.962+0000 7f95c223a700 1 -- 172.21.15.171:0/2651427187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95a40035c0 msgr2=0x7f95a4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:42.952 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.962+0000 7f95c223a700 1 --2- 172.21.15.171:0/2651427187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95a40035c0 0x7f95a4005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95bc1014d0 tx=0x7f95ac00feb0 comp rx=0 tx=0).stop 2024-03-26T17:53:42.952 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.962+0000 7f95c223a700 1 -- 172.21.15.171:0/2651427187 shutdown_connections 2024-03-26T17:53:42.952 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.962+0000 7f95c223a700 1 --2- 172.21.15.171:0/2651427187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95a40035c0 0x7f95a4005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:42.952 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.962+0000 7f95c223a700 1 -- 172.21.15.171:0/2651427187 >> 172.21.15.171:0/2651427187 conn(0x7f95bc0774a0 msgr2=0x7f95bc0778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:42.952 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.962+0000 7f95c223a700 1 -- 172.21.15.171:0/2651427187 shutdown_connections 2024-03-26T17:53:42.952 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.962+0000 7f95c223a700 1 -- 172.21.15.171:0/2651427187 wait complete. 2024-03-26T17:53:42.952 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95c223a700 1 Processor -- start 2024-03-26T17:53:42.952 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95c223a700 1 -- start start 2024-03-26T17:53:42.955 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95c223a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a40035c0 0x7f95bc19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:42.955 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95c223a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95bc19dd40 0x7f95bc1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:42.955 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95c223a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95bc197f00 0x7f95bc1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:42.955 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95c223a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc109fe0 con 0x7f95bc19dd40 2024-03-26T17:53:42.955 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95c223a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc109e60 con 0x7f95a40035c0 2024-03-26T17:53:42.955 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95c223a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc10a160 con 0x7f95bc197f00 2024-03-26T17:53:42.955 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95bbfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95bc197f00 0x7f95bc1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:42.955 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95bbfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95bc197f00 0x7f95bc1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:41598/0 (socket says 172.21.15.171:41598) 2024-03-26T17:53:42.955 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95bbfff700 1 -- 172.21.15.171:0/2955310489 learned_addr learned my addr 172.21.15.171:0/2955310489 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-26T17:53:42.955 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95baffd700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95bc19dd40 0x7f95bc1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95bbfff700 1 -- 172.21.15.171:0/2955310489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95bc197f00 msgr2=0x7f95bc1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95bbfff700 1 -- 172.21.15.171:0/2955310489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95bc197f00 msgr2=0x7f95bc1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95bbfff700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95bc197f00 0x7f95bc1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95bb7fe700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a40035c0 0x7f95bc19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.963+0000 7f95bbfff700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95bc197f00 0x7f95bc1983b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95baffd700 1 -- 172.21.15.171:0/2955310489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95bc197f00 msgr2=0x7f95bc1983b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95baffd700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95bc197f00 0x7f95bc1983b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95bb7fe700 1 -- 172.21.15.171:0/2955310489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a40035c0 msgr2=0x7f95bc19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95bb7fe700 1 -- 172.21.15.171:0/2955310489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a40035c0 msgr2=0x7f95bc19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95baffd700 1 -- 172.21.15.171:0/2955310489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a40035c0 msgr2=0x7f95bc19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95bb7fe700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a40035c0 0x7f95bc19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95bb7fe700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a40035c0 0x7f95bc19d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:53:42.956 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95baffd700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a40035c0 0x7f95bc19d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95baffd700 1 -- 172.21.15.171:0/2955310489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc1a46d0 con 0x7f95bc19dd40 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95baffd700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95bc19dd40 0x7f95bc1979c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95b000b550 tx=0x7f95b000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95b8ff9700 1 -- 172.21.15.171:0/2955310489 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0004230 con 0x7f95bc19dd40 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95b8ff9700 1 -- 172.21.15.171:0/2955310489 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b00043d0 con 0x7f95bc19dd40 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95c223a700 1 -- 172.21.15.171:0/2955310489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc1a4900 con 0x7f95bc19dd40 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95b8ff9700 1 -- 172.21.15.171:0/2955310489 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0011c20 con 0x7f95bc19dd40 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.964+0000 7f95c223a700 1 -- 172.21.15.171:0/2955310489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc1a4e40 con 0x7f95bc19dd40 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.965+0000 7f95b8ff9700 1 -- 172.21.15.171:0/2955310489 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f95b001a400 con 0x7f95bc19dd40 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.965+0000 7f95c223a700 1 -- 172.21.15.171:0/2955310489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95bc0639e0 con 0x7f95bc19dd40 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.965+0000 7f95b8ff9700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f95a80396b0 0x7f95a803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.965+0000 7f95b8ff9700 1 -- 172.21.15.171:0/2955310489 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f95b004bfa0 con 0x7f95bc19dd40 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.965+0000 7f95bb7fe700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f95a80396b0 0x7f95a803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:42.957 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.966+0000 7f95bb7fe700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f95a80396b0 0x7f95a803bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00f8d0 tx=0x7f95ac005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:42.960 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:42.971+0000 7f95b8ff9700 1 -- 172.21.15.171:0/2955310489 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f95b0020500 con 0x7f95bc19dd40 2024-03-26T17:53:43.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:43.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:43.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:43.246 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.257+0000 7f95c223a700 1 -- 172.21.15.171:0/2955310489 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f95bc02dc00 con 0x7f95bc19dd40 2024-03-26T17:53:43.246 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.257+0000 7f95b8ff9700 1 -- 172.21.15.171:0/2955310489 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f95b00202a0 con 0x7f95bc19dd40 2024-03-26T17:53:43.247 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-03-26T17:53:43.247 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":3,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","modified":"2024-03-26T17:53:37.069267Z","created":"2024-03-26T17:51:39.354207Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-26T17:53:43.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 -- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f95a80396b0 msgr2=0x7f95a803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:43.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f95a80396b0 0x7f95a803bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00f8d0 tx=0x7f95ac005a90 comp rx=0 tx=0).stop 2024-03-26T17:53:43.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 -- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95bc19dd40 msgr2=0x7f95bc1979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:43.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95bc19dd40 0x7f95bc1979c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95b000b550 tx=0x7f95b000b910 comp rx=0 tx=0).stop 2024-03-26T17:53:43.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 -- 172.21.15.171:0/2955310489 shutdown_connections 2024-03-26T17:53:43.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f95a80396b0 0x7f95a803bb70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:43.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95bc197f00 0x7f95bc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:43.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95a40035c0 0x7f95bc19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:43.250 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 --2- 172.21.15.171:0/2955310489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95bc19dd40 0x7f95bc1979c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:43.250 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 -- 172.21.15.171:0/2955310489 >> 172.21.15.171:0/2955310489 conn(0x7f95bc0774a0 msgr2=0x7f95bc106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:43.250 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 -- 172.21.15.171:0/2955310489 shutdown_connections 2024-03-26T17:53:43.250 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:43.260+0000 7f95c223a700 1 -- 172.21.15.171:0/2955310489 wait complete. 2024-03-26T17:53:43.250 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 3 2024-03-26T17:53:43.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:43.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:43.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:43.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:43.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:43.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:53:44.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:44 smithi171 ceph-mon[36033]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:44.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:44 smithi171 ceph-mon[36033]: from='client.? 172.21.15.171:0/2955310489' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T17:53:44.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:44.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:44.250 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-26T17:53:44.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-26T17:53:44.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:44 smithi115 ceph-mon[36174]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:44.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:44 smithi115 ceph-mon[36174]: from='client.? 172.21.15.171:0/2955310489' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T17:53:44.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:44.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:44.502 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:53:44.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:44 smithi049 ceph-mon[32331]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:44.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:44 smithi049 ceph-mon[32331]: from='client.? 172.21.15.171:0/2955310489' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T17:53:44.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:44.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:44.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.495 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T17:53:45.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:45.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:53:45.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:45.496 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:45 smithi115 ceph-mon[36174]: Reconfiguring daemon mon.a on smithi049 2024-03-26T17:53:45.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-26T17:53:45.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T17:53:45.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:45.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:53:45.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:45.511 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:45 smithi049 ceph-mon[32331]: Reconfiguring daemon mon.a on smithi049 2024-03-26T17:53:45.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-26T17:53:45.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-26T17:53:45.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-26T17:53:45.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:45.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T17:53:45.614 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:45.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:53:45.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:45.615 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:45 smithi171 ceph-mon[36033]: Reconfiguring daemon mon.a on smithi049 2024-03-26T17:53:46.510 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:46 smithi049 ceph-mon[32331]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:46.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:46 smithi171 ceph-mon[36033]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:46.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:46 smithi115 ceph-mon[36174]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:47.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.106+0000 7ffac69bc700 1 -- 172.21.15.49:0/2411604161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffac01065c0 msgr2=0x7ffac010e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.106+0000 7ffac69bc700 1 --2- 172.21.15.49:0/2411604161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffac01065c0 0x7ffac010e190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffab800b0d0 tx=0x7ffab800b3e0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.106+0000 7ffac69bc700 1 -- 172.21.15.49:0/2411604161 shutdown_connections 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.106+0000 7ffac69bc700 1 --2- 172.21.15.49:0/2411604161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffac01065c0 0x7ffac010e190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.106+0000 7ffac69bc700 1 --2- 172.21.15.49:0/2411604161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffac0101570 0x7ffac0101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.106+0000 7ffac69bc700 1 --2- 172.21.15.49:0/2411604161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffac0101f20 0x7ffac0105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.106+0000 7ffac69bc700 1 -- 172.21.15.49:0/2411604161 >> 172.21.15.49:0/2411604161 conn(0x7ffac0077300 msgr2=0x7ffac0077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.106+0000 7ffac69bc700 1 -- 172.21.15.49:0/2411604161 shutdown_connections 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.106+0000 7ffac69bc700 1 -- 172.21.15.49:0/2411604161 wait complete. 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.107+0000 7ffac69bc700 1 Processor -- start 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.107+0000 7ffac69bc700 1 -- start start 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.107+0000 7ffac69bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffac0101570 0x7ffac019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.107+0000 7ffac69bc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffac0101f20 0x7ffac019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.107+0000 7ffac69bc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffac01065c0 0x7ffac01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:47.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.107+0000 7ffac69bc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffac0111670 con 0x7ffac0101570 2024-03-26T17:53:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.107+0000 7ffac69bc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffac01114f0 con 0x7ffac01065c0 2024-03-26T17:53:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.107+0000 7ffac69bc700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffac01117f0 con 0x7ffac0101f20 2024-03-26T17:53:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffac0101570 0x7ffac019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffac0101570 0x7ffac019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42658/0 (socket says 172.21.15.49:42658) 2024-03-26T17:53:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabffff700 1 -- 172.21.15.49:0/25821172 learned_addr learned my addr 172.21.15.49:0/25821172 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:53:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffac4f59700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffac01065c0 0x7ffac01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabf7fe700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffac0101f20 0x7ffac019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabffff700 1 -- 172.21.15.49:0/25821172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffac0101f20 msgr2=0x7ffac019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabffff700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffac0101f20 0x7ffac019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabffff700 1 -- 172.21.15.49:0/25821172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffac01065c0 msgr2=0x7ffac01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabffff700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffac01065c0 0x7ffac01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabffff700 1 -- 172.21.15.49:0/25821172 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac0199150 con 0x7ffac0101570 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabffff700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffac0101570 0x7ffac019e830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffab000b530 tx=0x7ffab000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabd7fa700 1 -- 172.21.15.49:0/25821172 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0015020 con 0x7ffac0101570 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabd7fa700 1 -- 172.21.15.49:0/25821172 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffab0011d10 con 0x7ffac0101570 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffac69bc700 1 -- 172.21.15.49:0/25821172 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac0199440 con 0x7ffac0101570 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffabd7fa700 1 -- 172.21.15.49:0/25821172 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab000f2e0 con 0x7ffac0101570 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.108+0000 7ffac69bc700 1 -- 172.21.15.49:0/25821172 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac01a5700 con 0x7ffac0101570 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.109+0000 7ffabd7fa700 1 -- 172.21.15.49:0/25821172 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7ffab0018020 con 0x7ffac0101570 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.109+0000 7ffac69bc700 1 -- 172.21.15.49:0/25821172 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac0069070 con 0x7ffac0101570 2024-03-26T17:53:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.109+0000 7ffabd7fa700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ffaa8039460 0x7ffaa803b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.109+0000 7ffabd7fa700 1 -- 172.21.15.49:0/25821172 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffab004bcc0 con 0x7ffac0101570 2024-03-26T17:53:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.109+0000 7ffabf7fe700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ffaa8039460 0x7ffaa803b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:47.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.110+0000 7ffabf7fe700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ffaa8039460 0x7ffaa803b920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffab4005950 tx=0x7ffab40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:47.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.114+0000 7ffabd7fa700 1 -- 172.21.15.49:0/25821172 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffab001f020 con 0x7ffac0101570 2024-03-26T17:53:47.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.321+0000 7ffac69bc700 1 -- 172.21.15.49:0/25821172 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ffac002dc00 con 0x7ffac0101570 2024-03-26T17:53:47.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.322+0000 7ffabd7fa700 1 -- 172.21.15.49:0/25821172 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7ffab0022510 con 0x7ffac0101570 2024-03-26T17:53:47.314 INFO:teuthology.orchestra.run.smithi049.stdout:# minimal ceph.conf for 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:53:47.314 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-03-26T17:53:47.314 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T17:53:47.314 INFO:teuthology.orchestra.run.smithi049.stdout: mon_host = [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 -- 172.21.15.49:0/25821172 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ffaa8039460 msgr2=0x7ffaa803b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ffaa8039460 0x7ffaa803b920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffab4005950 tx=0x7ffab40058e0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 -- 172.21.15.49:0/25821172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffac0101570 msgr2=0x7ffac019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffac0101570 0x7ffac019e830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffab000b530 tx=0x7ffab000b840 comp rx=0 tx=0).stop 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 -- 172.21.15.49:0/25821172 shutdown_connections 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ffaa8039460 0x7ffaa803b920 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffac01065c0 0x7ffac01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffac0101570 0x7ffac019e830 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 --2- 172.21.15.49:0/25821172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffac0101f20 0x7ffac019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 -- 172.21.15.49:0/25821172 >> 172.21.15.49:0/25821172 conn(0x7ffac0077300 msgr2=0x7ffac006acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 -- 172.21.15.49:0/25821172 shutdown_connections 2024-03-26T17:53:47.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:47.324+0000 7ffac69bc700 1 -- 172.21.15.49:0/25821172 wait complete. 2024-03-26T17:53:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:47 smithi049 ceph-mon[32331]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:47 smithi049 ceph-mon[32331]: Reconfiguring mon.b (monmap changed)... 2024-03-26T17:53:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:53:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:47.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:47 smithi049 ceph-mon[32331]: Reconfiguring daemon mon.b on smithi115 2024-03-26T17:53:47.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:47 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/25821172' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:47.907 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-26T17:53:47.907 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:53:47.907 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T17:53:47.941 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:53:47.941 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:47.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:47 smithi115 ceph-mon[36174]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:47.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:47.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:47.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:47 smithi115 ceph-mon[36174]: Reconfiguring mon.b (monmap changed)... 2024-03-26T17:53:47.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:47.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:53:47.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:47.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:47 smithi115 ceph-mon[36174]: Reconfiguring daemon mon.b on smithi115 2024-03-26T17:53:47.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:47 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/25821172' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:48.012 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:53:48.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T17:53:48.044 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:53:48.044 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:48.112 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:53:48.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T17:53:48.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:47 smithi171 ceph-mon[36033]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:48.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:48.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:48.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:47 smithi171 ceph-mon[36033]: Reconfiguring mon.b (monmap changed)... 2024-03-26T17:53:48.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:48.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:53:48.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:48.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:47 smithi171 ceph-mon[36033]: Reconfiguring daemon mon.b on smithi115 2024-03-26T17:53:48.114 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:47 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/25821172' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:48.148 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:53:48.148 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:53:48.217 INFO:tasks.cephadm:Adding mgr.x on smithi049 2024-03-26T17:53:48.217 INFO:tasks.cephadm:Adding mgr.y on smithi115 2024-03-26T17:53:48.217 INFO:tasks.cephadm:Adding mgr.z on smithi171 2024-03-26T17:53:48.217 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi049=x;smithi115=y;smithi171=z' 2024-03-26T17:53:48.494 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:53:49.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:49 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:49 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:49 smithi171 ceph-mon[36033]: Reconfiguring mon.c (monmap changed)... 2024-03-26T17:53:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:49 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:49 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:53:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:49 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:49 smithi171 ceph-mon[36033]: Reconfiguring daemon mon.c on smithi171 2024-03-26T17:53:49.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:49 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:49.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:49 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:49.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:49 smithi115 ceph-mon[36174]: Reconfiguring mon.c (monmap changed)... 2024-03-26T17:53:49.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:49 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:49.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:49 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:53:49.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:49 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:49.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:49 smithi115 ceph-mon[36174]: Reconfiguring daemon mon.c on smithi171 2024-03-26T17:53:49.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:49 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:49.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:49 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:49.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:49 smithi049 ceph-mon[32331]: Reconfiguring mon.c (monmap changed)... 2024-03-26T17:53:49.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:49 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:53:49.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:49 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:53:49.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:49 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:49.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:49 smithi049 ceph-mon[32331]: Reconfiguring daemon mon.c on smithi171 2024-03-26T17:53:50.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:50 smithi171 ceph-mon[36033]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:50.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:50 smithi115 ceph-mon[36174]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:50.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:50 smithi049 ceph-mon[32331]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:50.919 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.929+0000 7f6ec62a6700 1 -- 172.21.15.171:0/680394533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ec0074d90 msgr2=0x7f6ec0075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.919 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.929+0000 7f6ec62a6700 1 --2- 172.21.15.171:0/680394533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ec0074d90 0x7f6ec0075210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009990 tx=0x7f6eb4009ca0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.929+0000 7f6ec62a6700 1 -- 172.21.15.171:0/680394533 shutdown_connections 2024-03-26T17:53:50.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.929+0000 7f6ec62a6700 1 --2- 172.21.15.171:0/680394533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ec010a2d0 0x7f6ec010c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.929+0000 7f6ec62a6700 1 --2- 172.21.15.171:0/680394533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec0076360 0x7f6ec00747c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.929+0000 7f6ec62a6700 1 --2- 172.21.15.171:0/680394533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ec0074d90 0x7f6ec0075210 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.929+0000 7f6ec62a6700 1 -- 172.21.15.171:0/680394533 >> 172.21.15.171:0/680394533 conn(0x7f6ec00fd990 msgr2=0x7f6ec00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:50.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.929+0000 7f6ec62a6700 1 -- 172.21.15.171:0/680394533 shutdown_connections 2024-03-26T17:53:50.920 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.929+0000 7f6ec62a6700 1 -- 172.21.15.171:0/680394533 wait complete. 2024-03-26T17:53:50.923 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ec62a6700 1 Processor -- start 2024-03-26T17:53:50.923 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ec62a6700 1 -- start start 2024-03-26T17:53:50.923 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ec62a6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ec0074d90 0x7f6ec019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ec62a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ec0076360 0x7f6ec019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ec62a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec010a2d0 0x7f6ec01a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ec62a6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01101e0 con 0x7f6ec0076360 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ec62a6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0110060 con 0x7f6ec010a2d0 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ec62a6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0110360 con 0x7f6ec0074d90 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ebffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ec0074d90 0x7f6ec019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ebffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ec0074d90 0x7f6ec019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:42782/0 (socket says 172.21.15.171:42782) 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ebffff700 1 -- 172.21.15.171:0/3326833479 learned_addr learned my addr 172.21.15.171:0/3326833479 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ebf7fe700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ec0076360 0x7f6ec019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.930+0000 7f6ec4843700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec010a2d0 0x7f6ec01a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.924 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ebffff700 1 -- 172.21.15.171:0/3326833479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec010a2d0 msgr2=0x7f6ec01a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ebffff700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec010a2d0 0x7f6ec01a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ebffff700 1 -- 172.21.15.171:0/3326833479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ec0076360 msgr2=0x7f6ec019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ebffff700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ec0076360 0x7f6ec019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ebffff700 1 -- 172.21.15.171:0/3326833479 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec01a2470 con 0x7f6ec0074d90 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ebffff700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ec0074d90 0x7f6ec019a810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000d9b0 tx=0x7f6eb000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ebd7fa700 1 -- 172.21.15.171:0/3326833479 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0017070 con 0x7f6ec0074d90 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ebd7fa700 1 -- 172.21.15.171:0/3326833479 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb0005020 con 0x7f6ec0074d90 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ec62a6700 1 -- 172.21.15.171:0/3326833479 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec01a2760 con 0x7f6ec0074d90 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ebd7fa700 1 -- 172.21.15.171:0/3326833479 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0009a90 con 0x7f6ec0074d90 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.931+0000 7f6ec62a6700 1 -- 172.21.15.171:0/3326833479 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec01a2ca0 con 0x7f6ec0074d90 2024-03-26T17:53:50.925 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.932+0000 7f6ebd7fa700 1 -- 172.21.15.171:0/3326833479 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f6eb0009c30 con 0x7f6ec0074d90 2024-03-26T17:53:50.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.932+0000 7f6ec62a6700 1 -- 172.21.15.171:0/3326833479 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec0069070 con 0x7f6ec0074d90 2024-03-26T17:53:50.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.932+0000 7f6ebd7fa700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6ea8039460 0x7f6ea803b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.932+0000 7f6ebd7fa700 1 -- 172.21.15.171:0/3326833479 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6eb004c320 con 0x7f6ec0074d90 2024-03-26T17:53:50.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.932+0000 7f6ebf7fe700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6ea8039460 0x7f6ea803b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.926 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.933+0000 7f6ebf7fe700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6ea8039460 0x7f6ea803b920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400b2e0 tx=0x7f6eb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:50.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:50.937+0000 7f6ebd7fa700 1 -- 172.21.15.171:0/3326833479 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6eb0020020 con 0x7f6ec0074d90 2024-03-26T17:53:51.134 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.144+0000 7f6ec62a6700 1 -- 172.21.15.171:0/3326833479 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi115=y;smithi171=z", "target": ["mon-mgr", ""]}) -- 0x7f6ec00627f0 con 0x7f6ea8039460 2024-03-26T17:53:51.145 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.155+0000 7f6ebd7fa700 1 -- 172.21.15.171:0/3326833479 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6ec00627f0 con 0x7f6ea8039460 2024-03-26T17:53:51.145 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mgr update... 2024-03-26T17:53:51.147 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 -- 172.21.15.171:0/3326833479 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6ea8039460 msgr2=0x7f6ea803b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.147 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6ea8039460 0x7f6ea803b920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400b2e0 tx=0x7f6eb4005fb0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.147 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 -- 172.21.15.171:0/3326833479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ec0074d90 msgr2=0x7f6ec019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.147 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ec0074d90 0x7f6ec019a810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000d9b0 tx=0x7f6eb000dcc0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.147 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 -- 172.21.15.171:0/3326833479 shutdown_connections 2024-03-26T17:53:51.147 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6ea8039460 0x7f6ea803b920 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.147 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec010a2d0 0x7f6ec01a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.147 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ec0074d90 0x7f6ec019a810 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.148 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 --2- 172.21.15.171:0/3326833479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ec0076360 0x7f6ec019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.148 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 -- 172.21.15.171:0/3326833479 >> 172.21.15.171:0/3326833479 conn(0x7f6ec00fd990 msgr2=0x7f6ec0106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.148 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 -- 172.21.15.171:0/3326833479 shutdown_connections 2024-03-26T17:53:51.148 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:53:51.157+0000 7f6ec62a6700 1 -- 172.21.15.171:0/3326833479 wait complete. 2024-03-26T17:53:51.702 DEBUG:teuthology.orchestra.run.smithi115:mgr.y> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.y.service 2024-03-26T17:53:51.705 DEBUG:teuthology.orchestra.run.smithi171:mgr.z> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.z.service 2024-03-26T17:53:51.708 INFO:tasks.cephadm:Deploying OSDs... 2024-03-26T17:53:51.708 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:53:51.708 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T17:53:51.726 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T17:53:51.726 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-03-26T17:53:51.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:51.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:51.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:51.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:53:51.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:53:51.732 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:53:51.732 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:51.732 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.732 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.732 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:51.732 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:51.732 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:51.732 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.733 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:53:51.733 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.733 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:53:51.733 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:53:51.733 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:53:51.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-26 17:52:56.406733212 +0000 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-26 17:47:47.137990364 +0000 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-26 17:47:47.137990364 +0000 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-26 17:47:47.137990364 +0000 2024-03-26T17:53:51.776 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T17:53:51.848 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-26T17:53:51.849 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-26T17:53:51.849 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000102841 s, 5.0 MB/s 2024-03-26T17:53:51.850 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T17:53:51.909 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-03-26T17:53:51.966 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T17:53:51.966 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:51.967 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-26T17:53:51.967 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:51.967 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:51.967 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-26 17:52:56.437732368 +0000 2024-03-26T17:53:51.967 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-26 17:47:47.479980943 +0000 2024-03-26T17:53:51.967 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-26 17:47:47.479980943 +0000 2024-03-26T17:53:51.967 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-26 17:47:47.479980943 +0000 2024-03-26T17:53:51.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T17:53:52.038 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-26T17:53:52.038 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-26T17:53:52.039 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000107343 s, 4.8 MB/s 2024-03-26T17:53:52.040 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T17:53:52.097 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-03-26T17:53:52.153 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T17:53:52.153 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:52.153 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-26T17:53:52.154 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:52.154 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:52.154 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-26 17:52:56.466731578 +0000 2024-03-26T17:53:52.154 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-26 17:47:47.829971303 +0000 2024-03-26T17:53:52.154 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-26 17:47:47.829971303 +0000 2024-03-26T17:53:52.154 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-26 17:47:47.829971303 +0000 2024-03-26T17:53:52.154 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T17:53:52.224 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-26T17:53:52.224 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-26T17:53:52.224 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.7308e-05 s, 5.3 MB/s 2024-03-26T17:53:52.226 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T17:53:52.282 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-03-26T17:53:52.339 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T17:53:52.339 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:52.339 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-26T17:53:52.339 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:52.339 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:52.339 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-03-26 17:52:56.492730870 +0000 2024-03-26T17:53:52.339 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-03-26 17:47:48.193961277 +0000 2024-03-26T17:53:52.339 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-03-26 17:47:48.193961277 +0000 2024-03-26T17:53:52.339 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-03-26 17:47:48.193961277 +0000 2024-03-26T17:53:52.340 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000114555 s, 4.5 MB/s 2024-03-26T17:53:52.411 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T17:53:52.468 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-03-26T17:53:52.468 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T17:53:52.487 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T17:53:52.487 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-03-26T17:53:52.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:52 smithi049 ceph-mon[32331]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:52.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:52 smithi049 ceph-mon[32331]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi115=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:53:52.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:52 smithi049 ceph-mon[32331]: Saving service mgr spec with placement smithi049=x;smithi115=y;smithi171=z;count:3 2024-03-26T17:53:52.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:52 smithi049 ceph-mon[32331]: Deploying daemon mgr.z on smithi171 2024-03-26T17:53:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T17:53:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:52.545 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-26T17:53:52.545 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:52.545 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:52.545 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-03-26 17:53:36.525105531 +0000 2024-03-26T17:53:52.545 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-03-26 17:47:50.642883705 +0000 2024-03-26T17:53:52.545 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-03-26 17:47:50.642883705 +0000 2024-03-26T17:53:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-03-26 17:47:50.642883705 +0000 2024-03-26T17:53:52.545 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T17:53:52.614 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-03-26T17:53:52.614 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-03-26T17:53:52.614 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000301277 s, 1.7 MB/s 2024-03-26T17:53:52.615 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T17:53:52.672 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-03-26T17:53:52.729 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T17:53:52.729 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:52.729 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-26T17:53:52.729 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:52.730 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:52.730 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-03-26 17:53:36.526105511 +0000 2024-03-26T17:53:52.730 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-03-26 17:47:50.980877044 +0000 2024-03-26T17:53:52.730 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-03-26 17:47:50.980877044 +0000 2024-03-26T17:53:52.730 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-03-26 17:47:50.980877044 +0000 2024-03-26T17:53:52.730 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T17:53:52.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:52 smithi115 ceph-mon[36174]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:52.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:52 smithi115 ceph-mon[36174]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi115=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:53:52.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:52 smithi115 ceph-mon[36174]: Saving service mgr spec with placement smithi049=x;smithi115=y;smithi171=z;count:3 2024-03-26T17:53:52.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:52 smithi115 ceph-mon[36174]: Deploying daemon mgr.z on smithi171 2024-03-26T17:53:52.770 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-03-26T17:53:52.770 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-03-26T17:53:52.770 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000196732 s, 2.6 MB/s 2024-03-26T17:53:52.771 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T17:53:52.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:52 smithi171 ceph-mon[36033]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:52.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:52 smithi171 ceph-mon[36033]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi115=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:53:52.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:52 smithi171 ceph-mon[36033]: Saving service mgr spec with placement smithi049=x;smithi115=y;smithi171=z;count:3 2024-03-26T17:53:52.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:52 smithi171 ceph-mon[36033]: Deploying daemon mgr.z on smithi171 2024-03-26T17:53:52.827 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-03-26T17:53:52.884 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T17:53:52.884 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:52.884 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-03-26T17:53:52.884 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:52.884 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:52.884 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-03-26 17:53:36.526105511 +0000 2024-03-26T17:53:52.884 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-03-26 17:47:51.314870462 +0000 2024-03-26T17:53:52.884 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-03-26 17:47:51.314870462 +0000 2024-03-26T17:53:52.884 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-03-26 17:47:51.314870462 +0000 2024-03-26T17:53:52.884 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T17:53:52.950 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-03-26T17:53:52.950 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-03-26T17:53:52.950 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000225788 s, 2.3 MB/s 2024-03-26T17:53:52.952 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T17:53:53.008 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-03-26T17:53:53.064 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T17:53:53.064 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:53.064 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-26T17:53:53.064 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:53.064 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:53.065 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-03-26 17:53:36.527105492 +0000 2024-03-26T17:53:53.065 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-03-26 17:47:51.642863998 +0000 2024-03-26T17:53:53.065 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-03-26 17:47:51.642863998 +0000 2024-03-26T17:53:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-03-26 17:47:51.642863998 +0000 2024-03-26T17:53:53.065 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T17:53:53.130 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-03-26T17:53:53.130 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-03-26T17:53:53.130 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000170386 s, 3.0 MB/s 2024-03-26T17:53:53.131 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T17:53:53.186 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-03-26T17:53:53.186 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T17:53:53.205 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T17:53:53.205 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-03-26T17:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T17:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-26T17:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-03-26 17:53:53.103895676 +0000 2024-03-26T17:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-03-26 17:48:09.315623233 +0000 2024-03-26T17:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-03-26 17:48:09.315623233 +0000 2024-03-26T17:53:53.261 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-03-26 17:48:09.315623233 +0000 2024-03-26T17:53:53.262 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T17:53:53.329 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-03-26T17:53:53.330 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-03-26T17:53:53.330 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000273344 s, 1.9 MB/s 2024-03-26T17:53:53.332 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T17:53:53.388 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-03-26T17:53:53.401 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:53 smithi171 podman[37690]: 2024-03-26 17:53:53.291657244 +0000 UTC m=+0.017969558 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:53:53.401 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:53 smithi171 podman[37690]: 2024-03-26 17:53:53.412525901 +0000 UTC m=+0.138838258 container create ecd495246d5c9d34b16331e061c1c346bce3470162aa894e49c1c5759c77d723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=) 2024-03-26T17:53:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T17:53:53.416 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:53.417 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-26T17:53:53.417 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:53.417 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:53.417 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-03-26 17:53:53.103895676 +0000 2024-03-26T17:53:53.417 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-03-26 17:48:09.675616052 +0000 2024-03-26T17:53:53.417 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-03-26 17:48:09.675616052 +0000 2024-03-26T17:53:53.418 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-03-26 17:48:09.675616052 +0000 2024-03-26T17:53:53.419 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T17:53:53.484 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-03-26T17:53:53.484 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-03-26T17:53:53.484 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000229532 s, 2.2 MB/s 2024-03-26T17:53:53.485 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T17:53:53.541 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-03-26T17:53:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T17:53:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:53.596 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-03-26T17:53:53.596 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:53.596 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:53.596 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-03-26 17:53:53.103895676 +0000 2024-03-26T17:53:53.596 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-03-26 17:48:10.005609470 +0000 2024-03-26T17:53:53.596 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-03-26 17:48:10.005609470 +0000 2024-03-26T17:53:53.596 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-03-26 17:48:10.005609470 +0000 2024-03-26T17:53:53.597 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T17:53:53.658 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:53 smithi171 podman[37690]: 2024-03-26 17:53:53.640717721 +0000 UTC m=+0.367030047 container init ecd495246d5c9d34b16331e061c1c346bce3470162aa894e49c1c5759c77d723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-26T17:53:53.658 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:53 smithi171 podman[37690]: 2024-03-26 17:53:53.644093805 +0000 UTC m=+0.370406107 container start ecd495246d5c9d34b16331e061c1c346bce3470162aa894e49c1c5759c77d723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True) 2024-03-26T17:53:53.664 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-03-26T17:53:53.664 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-03-26T17:53:53.664 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000184517 s, 2.8 MB/s 2024-03-26T17:53:53.665 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T17:53:53.721 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-03-26T17:53:53.777 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T17:53:53.778 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:53:53.778 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-26T17:53:53.778 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:53:53.778 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:53:53.778 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-03-26 17:53:53.103895676 +0000 2024-03-26T17:53:53.778 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-03-26 17:48:10.342602748 +0000 2024-03-26T17:53:53.778 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-03-26 17:48:10.342602748 +0000 2024-03-26T17:53:53.778 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-03-26 17:48:10.342602748 +0000 2024-03-26T17:53:53.779 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T17:53:53.845 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-03-26T17:53:53.845 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-03-26T17:53:53.845 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000198705 s, 2.6 MB/s 2024-03-26T17:53:53.847 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T17:53:53.903 INFO:tasks.cephadm:Deploying osd.0 on smithi049 with /dev/vg_nvme/lv_4... 2024-03-26T17:53:53.903 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T17:53:54.113 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:53 smithi171 ceph-mgr[37825]: -- 172.21.15.171:0/1986645663 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a6ae0c5a0 con 0x558a6ade1000 2024-03-26T17:53:54.113 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:53 smithi171 bash[37690]: ecd495246d5c9d34b16331e061c1c346bce3470162aa894e49c1c5759c77d723 2024-03-26T17:53:54.113 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:53 smithi171 systemd[1]: Started Ceph mgr.z for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:53:54.155 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:53:54.474 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:54.203+0000 7fd91c4bd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T17:53:54.474 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:54.296+0000 7fd91c4bd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T17:53:54.582 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:54 smithi049 ceph-mon[32331]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:54.582 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:54.582 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:54.582 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:54.582 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:53:54.583 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:53:54.583 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:53:54.583 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:54.704 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:54 smithi115 ceph-mon[36174]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:54.704 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:54.704 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:54.704 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:54.704 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:53:54.704 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:53:54.705 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:53:54.705 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:54.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-mon[36033]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:54.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:54.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:54.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:54.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:53:54.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:53:54.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:53:54.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:55.249 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:54 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:54.938+0000 7fd91c4bd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T17:53:55.249 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:55.028+0000 7fd91c4bd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T17:53:55.249 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:55.166+0000 7fd91c4bd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T17:53:55.568 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:55 smithi115 ceph-mon[36174]: Deploying daemon mgr.y on smithi115 2024-03-26T17:53:55.587 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:55 smithi171 ceph-mon[36033]: Deploying daemon mgr.y on smithi115 2024-03-26T17:53:55.588 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:55.258+0000 7fd91c4bd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T17:53:55.588 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:55.339+0000 7fd91c4bd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T17:53:55.588 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:55.597+0000 7fd91c4bd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T17:53:55.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:55 smithi049 ceph-mon[32331]: Deploying daemon mgr.y on smithi115 2024-03-26T17:53:55.863 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:55.703+0000 7fd91c4bd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T17:53:55.863 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:55.785+0000 7fd91c4bd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T17:53:55.926 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:55 smithi115 systemd[1]: Starting Ceph mgr.y for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:53:55.926 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:55 smithi115 podman[37541]: 2024-03-26 17:53:55.937247243 +0000 UTC m=+0.089283065 container create e3e1ac33f56396f3c649d282d674eeec7bf9113553c6a1c6763905b361ab141a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-26T17:53:56.066 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T17:53:56.066 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T17:53:56.066 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-26T17:53:56.066 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-26T17:53:56.066 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00811223 s, 1.3 GB/s 2024-03-26T17:53:56.066 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 2024-03-26T17:53:56.067 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-26T17:53:56.180 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:55 smithi115 podman[37541]: 2024-03-26 17:53:55.865695097 +0000 UTC m=+0.017730926 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:53:56.180 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:56 smithi115 podman[37541]: 2024-03-26 17:53:56.164427487 +0000 UTC m=+0.316463289 container init e3e1ac33f56396f3c649d282d674eeec7bf9113553c6a1c6763905b361ab141a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-26T17:53:56.180 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:56 smithi115 podman[37541]: 2024-03-26 17:53:56.167187101 +0000 UTC m=+0.319222905 container start e3e1ac33f56396f3c649d282d674eeec7bf9113553c6a1c6763905b361ab141a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T17:53:56.362 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:55.877+0000 7fd91c4bd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T17:53:56.363 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:56 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:56.052+0000 7fd91c4bd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T17:53:56.480 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:56 smithi115 ceph-mgr[37560]: -- 172.21.15.115:0/1655977734 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56422f6925a0 con 0x56422f666c00 2024-03-26T17:53:56.480 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:56 smithi115 bash[37541]: e3e1ac33f56396f3c649d282d674eeec7bf9113553c6a1c6763905b361ab141a 2024-03-26T17:53:56.481 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:56 smithi115 systemd[1]: Started Ceph mgr.y for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:53:56.684 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:56 smithi049 ceph-mon[32331]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:56.684 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:56 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.684 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:56 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.684 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:56 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.684 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:56 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.710 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_4 2024-03-26T17:53:56.743 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:56.723+0000 7fb2f9c81200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T17:53:56.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:56 smithi115 ceph-mon[36174]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:56.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:56 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:56 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:56 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:56 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:56 smithi171 ceph-mon[36033]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:56.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:56 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:56 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:56 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:56 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:56.955 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:53:57.244 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:56.820+0000 7fb2f9c81200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T17:53:57.339 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:57.050+0000 7fd91c4bd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T17:53:57.339 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:57.192+0000 7fd91c4bd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T17:53:57.597 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:57.598 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:57.516+0000 7fb2f9c81200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T17:53:57.598 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:57.608+0000 7fb2f9c81200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T17:53:57.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:57.613 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:57.538+0000 7fd91c4bd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T17:53:57.715 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:57.848 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:57.756+0000 7fb2f9c81200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T17:53:57.848 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:57.858+0000 7fb2f9c81200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T17:53:58.006 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:57.628+0000 7fd91c4bd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T17:53:58.006 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:57.832+0000 7fd91c4bd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:53:58.006 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:58.017+0000 7fd91c4bd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T17:53:58.192 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:57.942+0000 7fb2f9c81200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T17:53:58.192 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:58.202+0000 7fb2f9c81200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T17:53:58.475 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:58.310+0000 7fb2f9c81200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T17:53:58.476 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:58.394+0000 7fb2f9c81200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T17:53:58.476 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:58.486+0000 7fb2f9c81200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T17:53:58.548 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:58.297+0000 7fd91c4bd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:53:58.548 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:58.559+0000 7fd91c4bd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:53:58.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:53:58.550 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:58.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 -- 172.21.15.49:0/3462200147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60a0074db0 msgr2=0x7f60a0075230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:58.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 --2- 172.21.15.49:0/3462200147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60a0074db0 0x7f60a0075230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6090009990 tx=0x7f6090009ca0 comp rx=0 tx=0).stop 2024-03-26T17:53:58.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 -- 172.21.15.49:0/3462200147 shutdown_connections 2024-03-26T17:53:58.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 --2- 172.21.15.49:0/3462200147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60a010a2e0 0x7f60a010c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:58.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 --2- 172.21.15.49:0/3462200147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60a0076380 0x7f60a00747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:58.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 --2- 172.21.15.49:0/3462200147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60a0074db0 0x7f60a0075230 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:58.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 -- 172.21.15.49:0/3462200147 >> 172.21.15.49:0/3462200147 conn(0x7f60a00fd9c0 msgr2=0x7f60a00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:58.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 -- 172.21.15.49:0/3462200147 shutdown_connections 2024-03-26T17:53:58.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 -- 172.21.15.49:0/3462200147 wait complete. 2024-03-26T17:53:58.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 Processor -- start 2024-03-26T17:53:58.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.572+0000 7f60a75f4700 1 -- start start 2024-03-26T17:53:58.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a75f4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60a0074db0 0x7f60a019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a75f4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60a0076380 0x7f60a019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a75f4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60a010a2e0 0x7f60a01a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a75f4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60a01101d0 con 0x7f60a0076380 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a75f4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0110050 con 0x7f60a010a2e0 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a75f4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0110350 con 0x7f60a0074db0 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a4b8f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60a0076380 0x7f60a019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a4b8f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60a0076380 0x7f60a019acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46800/0 (socket says 172.21.15.49:46800) 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a5390700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60a0074db0 0x7f60a019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a4b8f700 1 -- 172.21.15.49:0/1192293907 learned_addr learned my addr 172.21.15.49:0/1192293907 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a5b91700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60a010a2e0 0x7f60a01a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a5390700 1 -- 172.21.15.49:0/1192293907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60a010a2e0 msgr2=0x7f60a01a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:58.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a5390700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60a010a2e0 0x7f60a01a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a5390700 1 -- 172.21.15.49:0/1192293907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60a0076380 msgr2=0x7f60a019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a5390700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60a0076380 0x7f60a019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a5390700 1 -- 172.21.15.49:0/1192293907 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a01a2410 con 0x7f60a0074db0 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a4b8f700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60a0076380 0x7f60a019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.573+0000 7f60a5390700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60a0074db0 0x7f60a019a7b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f609c00ea20 tx=0x7f609c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.574+0000 7f60967fc700 1 -- 172.21.15.49:0/1192293907 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c00cbd0 con 0x7f60a0074db0 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.574+0000 7f60967fc700 1 -- 172.21.15.49:0/1192293907 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609c00ee60 con 0x7f60a0074db0 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.574+0000 7f60967fc700 1 -- 172.21.15.49:0/1192293907 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c018680 con 0x7f60a0074db0 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.574+0000 7f60a75f4700 1 -- 172.21.15.49:0/1192293907 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a01a2700 con 0x7f60a0074db0 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.574+0000 7f60a75f4700 1 -- 172.21.15.49:0/1192293907 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a01a2bf0 con 0x7f60a0074db0 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.575+0000 7f60967fc700 1 -- 172.21.15.49:0/1192293907 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 13) ==== 46350+0+0 (secure 0 0 0) 0x7f609c018820 con 0x7f60a0074db0 2024-03-26T17:53:58.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.575+0000 7f60a75f4700 1 -- 172.21.15.49:0/1192293907 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a0069070 con 0x7f60a0074db0 2024-03-26T17:53:58.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.575+0000 7f60967fc700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f608c039460 0x7f608c03b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:58.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.575+0000 7f60967fc700 1 -- 172.21.15.49:0/1192293907 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f609c04c880 con 0x7f60a0074db0 2024-03-26T17:53:58.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.575+0000 7f60a4b8f700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f608c039460 0x7f608c03b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:58.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.575+0000 7f60a4b8f700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f608c039460 0x7f608c03b920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f609000b2e0 tx=0x7f609000b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:58.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.580+0000 7f60967fc700 1 -- 172.21.15.49:0/1192293907 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f609c00fc70 con 0x7f60a0074db0 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:53:58.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:58.743 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:58.660+0000 7fb2f9c81200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T17:53:58.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:53:58.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:58.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:58.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:53:58.745 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:53:58.745 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:53:58.745 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:58.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:53:58.799+0000 7f60a75f4700 1 -- 172.21.15.49:0/1192293907 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f60a00627f0 con 0x7f608c039460 2024-03-26T17:53:58.863 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:58.795+0000 7fd91c4bd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T17:53:59.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:59 smithi049 ceph-mon[32331]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T17:53:59.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:59 smithi049 ceph-mon[32331]: Reconfiguring daemon mgr.x on smithi049 2024-03-26T17:53:59.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:53:59.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:53:59.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:53:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:59.741 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:59.622+0000 7fb2f9c81200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T17:53:59.741 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:53:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:53:59.751+0000 7fb2f9c81200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T17:53:59.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:59 smithi115 ceph-mon[36174]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T17:53:59.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:59 smithi115 ceph-mon[36174]: Reconfiguring daemon mgr.x on smithi049 2024-03-26T17:53:59.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:59 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:53:59.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:59 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:53:59.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:53:59 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:59.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:59 smithi171 ceph-mon[36033]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T17:53:59.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:59 smithi171 ceph-mon[36033]: Reconfiguring daemon mgr.x on smithi049 2024-03-26T17:53:59.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:59 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:53:59.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:59 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:53:59.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:53:59 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:59.863 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:53:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:53:59.591+0000 7fd91c4bd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T17:54:00.351 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:00.082+0000 7fb2f9c81200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T17:54:00.351 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:00.169+0000 7fb2f9c81200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T17:54:00.351 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:00.361+0000 7fb2f9c81200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:54:00.488 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:54:00.191+0000 7fd91c4bd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T17:54:00.489 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:54:00.287+0000 7fd91c4bd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T17:54:00.489 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T17:54:00.499+0000 7fd91c4bd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T17:54:00.743 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:00.546+0000 7fb2f9c81200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T17:54:00.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-mon[36174]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:00.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-mon[36174]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:54:00.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:00.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:00.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:00.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:00.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:00.745 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:00.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:00 smithi049 ceph-mon[32331]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:00.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:00 smithi049 ceph-mon[32331]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:54:00.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:00 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:00.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:00 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:00.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:00 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:00.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:00 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:00.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:00 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:00.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:00 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:00.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-mon[36033]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:00.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-mon[36033]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:54:00.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:00.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:00.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:00.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:00.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:00.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:00 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:01.067 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:00.824+0000 7fb2f9c81200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:54:01.067 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:01.077+0000 7fb2f9c81200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T17:54:01.493 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:01.309+0000 7fb2f9c81200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T17:54:01.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:01.511+0000 7f60967fc700 1 -- 172.21.15.49:0/1192293907 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mgrmap(e 14) ==== 92074+0+0 (secure 0 0 0) 0x7f609c04ecf0 con 0x7f60a0074db0 2024-03-26T17:54:01.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:01 smithi049 ceph-mon[32331]: Standby manager daemon z started 2024-03-26T17:54:01.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:01 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T17:54:01.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:01 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:54:01.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:01 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T17:54:01.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:01 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:54:01.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:01 smithi171 ceph-mon[36033]: Standby manager daemon z started 2024-03-26T17:54:01.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:01 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T17:54:01.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:01 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:54:01.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:01 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T17:54:01.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:01 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:54:01.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:01 smithi115 ceph-mon[36174]: Standby manager daemon z started 2024-03-26T17:54:01.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:01 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T17:54:01.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:01 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:54:01.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:01 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T17:54:01.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:01 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.171:0/3281017552' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:54:02.493 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:02.105+0000 7fb2f9c81200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T17:54:02.755 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:02.669+0000 7fb2f9c81200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T17:54:02.755 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:02.766+0000 7fb2f9c81200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T17:54:02.756 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-mon[36174]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:02.756 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-mon[36174]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-26T17:54:02.756 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T17:54:02.756 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1307564014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22af3c21-ba61-4714-bf75-a3d2761bed59"}]: dispatch 2024-03-26T17:54:02.756 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1307564014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22af3c21-ba61-4714-bf75-a3d2761bed59"}]': finished 2024-03-26T17:54:02.756 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-mon[36174]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T17:54:02.756 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:02.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:02 smithi049 ceph-mon[32331]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:02.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:02 smithi049 ceph-mon[32331]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-26T17:54:02.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:02 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T17:54:02.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:02 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1307564014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22af3c21-ba61-4714-bf75-a3d2761bed59"}]: dispatch 2024-03-26T17:54:02.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:02 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1307564014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22af3c21-ba61-4714-bf75-a3d2761bed59"}]': finished 2024-03-26T17:54:02.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:02 smithi049 ceph-mon[32331]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T17:54:02.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:02 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:02.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:02 smithi171 ceph-mon[36033]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:02.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:02 smithi171 ceph-mon[36033]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-26T17:54:02.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:02 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T17:54:02.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:02 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1307564014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22af3c21-ba61-4714-bf75-a3d2761bed59"}]: dispatch 2024-03-26T17:54:02.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:02 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1307564014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22af3c21-ba61-4714-bf75-a3d2761bed59"}]': finished 2024-03-26T17:54:02.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:02 smithi171 ceph-mon[36033]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T17:54:02.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:02 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:03.243 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 17:54:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T17:54:02.979+0000 7fb2f9c81200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T17:54:03.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:03.517+0000 7f60967fc700 1 -- 172.21.15.49:0/1192293907 <== mon.1 v2:172.21.15.171:3300/0 8 ==== mgrmap(e 15) ==== 137798+0+0 (secure 0 0 0) 0x7f609c018c10 con 0x7f60a0074db0 2024-03-26T17:54:03.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:03 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2830499321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:54:03.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:03 smithi049 ceph-mon[32331]: Standby manager daemon y started 2024-03-26T17:54:03.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:03 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T17:54:03.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:03 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:54:03.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:03 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T17:54:03.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:03 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:54:03.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:03 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2830499321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:54:03.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:03 smithi171 ceph-mon[36033]: Standby manager daemon y started 2024-03-26T17:54:03.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:03 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T17:54:03.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:03 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:54:03.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:03 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T17:54:03.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:03 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:54:03.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:03 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2830499321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:54:03.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:03 smithi115 ceph-mon[36174]: Standby manager daemon y started 2024-03-26T17:54:03.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:03 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T17:54:03.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:03 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:54:03.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:03 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T17:54:03.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:03 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.115:0/3727302309' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:54:04.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:04 smithi115 ceph-mon[36174]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:04.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:04 smithi115 ceph-mon[36174]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-26T17:54:04.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:04 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T17:54:04.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:04 smithi049 ceph-mon[32331]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:04.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:04 smithi049 ceph-mon[32331]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-26T17:54:04.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:04 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T17:54:04.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:04 smithi171 ceph-mon[36033]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:04.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:04 smithi171 ceph-mon[36033]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-26T17:54:04.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:04 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T17:54:06.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:06 smithi049 ceph-mon[32331]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:06.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:06 smithi171 ceph-mon[36033]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:06.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:06 smithi115 ceph-mon[36174]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:08.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:08 smithi171 ceph-mon[36033]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:08.864 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:08 smithi049 ceph-mon[32331]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:08.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:08 smithi115 ceph-mon[36174]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:09.832 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T17:54:09.833 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:09.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T17:54:09.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:09.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T17:54:09.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:10.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:10 smithi049 ceph-mon[32331]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:10.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:10 smithi049 ceph-mon[32331]: Deploying daemon osd.0 on smithi049 2024-03-26T17:54:10.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:10 smithi171 ceph-mon[36033]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:10.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:10 smithi171 ceph-mon[36033]: Deploying daemon osd.0 on smithi049 2024-03-26T17:54:10.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:10 smithi115 ceph-mon[36174]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:10.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:10 smithi115 ceph-mon[36174]: Deploying daemon osd.0 on smithi049 2024-03-26T17:54:12.813 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:12 smithi049 ceph-mon[32331]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:12.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:12 smithi171 ceph-mon[36033]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:12.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:12 smithi115 ceph-mon[36174]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:14.761 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:14 smithi049 ceph-mon[32331]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:14.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:14 smithi171 ceph-mon[36033]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:14.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:14 smithi115 ceph-mon[36174]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:15.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:15 smithi115 ceph-mon[36174]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:15.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:15.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:15.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:15 smithi049 ceph-mon[32331]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:15 smithi171 ceph-mon[36033]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:18.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:18 smithi115 ceph-mon[36174]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:18.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:18 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:18.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:18 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:18.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:18 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:18.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:18 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:18.494 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:18 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:18.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:18 smithi049 ceph-mon[32331]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:18.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:18 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:18.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:18 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:18.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:18 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:18.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:18 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:18.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:18 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:18.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:18 smithi171 ceph-mon[36033]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:18.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:18 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:18.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:18 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:18.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:18 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:18.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:18 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:18.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:18 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:18.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.623+0000 7f60967fc700 1 -- 172.21.15.49:0/1192293907 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f60a00627f0 con 0x7f608c039460 2024-03-26T17:54:18.613 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 0 on host 'smithi049' 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 -- 172.21.15.49:0/1192293907 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f608c039460 msgr2=0x7f608c03b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f608c039460 0x7f608c03b920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f609000b2e0 tx=0x7f609000b880 comp rx=0 tx=0).stop 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 -- 172.21.15.49:0/1192293907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60a0074db0 msgr2=0x7f60a019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60a0074db0 0x7f60a019a7b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f609c00ea20 tx=0x7f609c00ed30 comp rx=0 tx=0).stop 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 -- 172.21.15.49:0/1192293907 shutdown_connections 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f608c039460 0x7f608c03b920 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60a010a2e0 0x7f60a01a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60a0074db0 0x7f60a019a7b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 --2- 172.21.15.49:0/1192293907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60a0076380 0x7f60a019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 -- 172.21.15.49:0/1192293907 >> 172.21.15.49:0/1192293907 conn(0x7f60a00fd9c0 msgr2=0x7f60a0106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.626+0000 7f60a75f4700 1 -- 172.21.15.49:0/1192293907 shutdown_connections 2024-03-26T17:54:18.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:54:18.627+0000 7f60a75f4700 1 -- 172.21.15.49:0/1192293907 wait complete. 2024-03-26T17:54:19.321 DEBUG:teuthology.orchestra.run.smithi049:osd.0> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.0.service 2024-03-26T17:54:19.323 INFO:tasks.cephadm:Deploying osd.1 on smithi115 with /dev/vg_nvme/lv_4... 2024-03-26T17:54:19.323 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T17:54:19.487 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:19 smithi049 ceph-mon[32331]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T17:54:19.487 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:19.487 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:19.487 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:19.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:19 smithi115 ceph-mon[36174]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T17:54:19.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:19.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:19.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:19.566 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:54:19.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:19 smithi171 ceph-mon[36033]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T17:54:19.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:19.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:19.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:20.341 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:20 smithi115 ceph-mon[36174]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:20.341 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:20 smithi115 ceph-mon[36174]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T17:54:20.341 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:20 smithi115 ceph-mon[36174]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T17:54:20.341 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:20 smithi115 ceph-mon[36174]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:54:20.341 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:20 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:20.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:20 smithi049 ceph-mon[32331]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:20.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:20 smithi049 ceph-mon[32331]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T17:54:20.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:20 smithi049 ceph-mon[32331]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T17:54:20.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:20 smithi049 ceph-mon[32331]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:54:20.509 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:20 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:20.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:20 smithi171 ceph-mon[36033]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:20.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:20 smithi171 ceph-mon[36033]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T17:54:20.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:20 smithi171 ceph-mon[36033]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T17:54:20.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:20 smithi171 ceph-mon[36033]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:54:20.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:20 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:20.866 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 17:54:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0[46715]: 2024-03-26T17:54:20.685+0000 7f81fe32d700 -1 osd.0 0 waiting for initial osdmap 2024-03-26T17:54:20.867 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 17:54:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0[46715]: 2024-03-26T17:54:20.688+0000 7f81f6118700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:54:21.417 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T17:54:21.417 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T17:54:21.417 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-03-26T17:54:21.417 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-03-26T17:54:21.417 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112611 s, 931 MB/s 2024-03-26T17:54:21.417 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-03-26T17:54:21.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:21 smithi115 ceph-mon[36174]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:54:21.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:21 smithi115 ceph-mon[36174]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T17:54:21.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:21 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:21.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:21 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:21.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:21 smithi115 ceph-mon[36174]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' 2024-03-26T17:54:21.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:21 smithi049 ceph-mon[32331]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:54:21.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:21 smithi049 ceph-mon[32331]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T17:54:21.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:21.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:21.508 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:21 smithi049 ceph-mon[32331]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' 2024-03-26T17:54:21.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:21 smithi171 ceph-mon[36033]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:54:21.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:21 smithi171 ceph-mon[36033]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T17:54:21.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:21 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:21.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:21 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:21.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:21 smithi171 ceph-mon[36033]: from='osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781]' entity='osd.0' 2024-03-26T17:54:22.026 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2024-03-26T17:54:22.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:22 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:54:22.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:22 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:54:22.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:22 smithi049 ceph-mon[32331]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:22.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:22.258 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:22 smithi049 ceph-mon[32331]: osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] boot 2024-03-26T17:54:22.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:22 smithi049 ceph-mon[32331]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T17:54:22.259 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:22.268 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:54:22.373 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:22 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:54:22.374 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:22 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:54:22.374 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:22 smithi115 ceph-mon[36174]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:22.374 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:22.374 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:22 smithi115 ceph-mon[36174]: osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] boot 2024-03-26T17:54:22.374 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:22 smithi115 ceph-mon[36174]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T17:54:22.374 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:22.513 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:22 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:54:22.513 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:22 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:54:22.514 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:22 smithi171 ceph-mon[36033]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:54:22.514 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:22.514 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:22 smithi171 ceph-mon[36033]: osd.0 [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] boot 2024-03-26T17:54:22.514 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:22 smithi171 ceph-mon[36033]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T17:54:22.514 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:54:23.475 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:23 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi049 2024-03-26T17:54:23.476 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:23 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.476 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:23 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.476 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:23 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:54:23.476 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:23 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi049 to 17088M 2024-03-26T17:54:23.476 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:23 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.476 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:23 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:23.476 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:23 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:23.476 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:23 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:23 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi049 2024-03-26T17:54:23.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:23 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:23 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:23 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:54:23.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:23 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi049 to 17088M 2024-03-26T17:54:23.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:23 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:23 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:23.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:23 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:23.613 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:23 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.685 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 -- 172.21.15.115:0/3156065760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff10c102ef0 msgr2=0x7ff10c1052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 --2- 172.21.15.115:0/3156065760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff10c102ef0 0x7ff10c1052e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff108009990 tx=0x7ff108009ca0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 -- 172.21.15.115:0/3156065760 shutdown_connections 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 --2- 172.21.15.115:0/3156065760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c108150 0x7ff10c10a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 --2- 172.21.15.115:0/3156065760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff10c102ef0 0x7ff10c1052e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 --2- 172.21.15.115:0/3156065760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c105820 0x7ff10c107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 -- 172.21.15.115:0/3156065760 >> 172.21.15.115:0/3156065760 conn(0x7ff10c0fc8a0 msgr2=0x7ff10c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 -- 172.21.15.115:0/3156065760 shutdown_connections 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 -- 172.21.15.115:0/3156065760 wait complete. 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 Processor -- start 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 -- start start 2024-03-26T17:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff10c102ef0 0x7ff10c102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.695+0000 7ff1133df700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c105820 0x7ff10c100ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff1133df700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c108150 0x7ff10c1013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff1133df700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c10dff0 con 0x7ff10c108150 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff1133df700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c10de70 con 0x7ff10c102ef0 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff1133df700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c10e170 con 0x7ff10c105820 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11117b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff10c102ef0 0x7ff10c102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11117b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff10c102ef0 0x7ff10c102850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38364/0 (socket says 172.21.15.115:38364) 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11117b700 1 -- 172.21.15.115:0/400927573 learned_addr learned my addr 172.21.15.115:0/400927573 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11097a700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c105820 0x7ff10c100ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11117b700 1 -- 172.21.15.115:0/400927573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c105820 msgr2=0x7ff10c100ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11117b700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c105820 0x7ff10c100ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11197c700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c108150 0x7ff10c1013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11117b700 1 -- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c108150 msgr2=0x7ff10c1013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11117b700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c108150 0x7ff10c1013e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11117b700 1 -- 172.21.15.115:0/400927573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10c101c90 con 0x7ff10c102ef0 2024-03-26T17:54:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11197c700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c108150 0x7ff10c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:54:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff11117b700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff10c102ef0 0x7ff10c102850 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff108005910 tx=0x7ff10800bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff1027fc700 1 -- 172.21.15.115:0/400927573 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108026070 con 0x7ff10c102ef0 2024-03-26T17:54:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff1133df700 1 -- 172.21.15.115:0/400927573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10c1a2750 con 0x7ff10c102ef0 2024-03-26T17:54:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff1027fc700 1 -- 172.21.15.115:0/400927573 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff10800f460 con 0x7ff10c102ef0 2024-03-26T17:54:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff1027fc700 1 -- 172.21.15.115:0/400927573 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108021670 con 0x7ff10c102ef0 2024-03-26T17:54:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.696+0000 7ff1133df700 1 -- 172.21.15.115:0/400927573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10c1a2b50 con 0x7ff10c102ef0 2024-03-26T17:54:23.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.697+0000 7ff1133df700 1 -- 172.21.15.115:0/400927573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff10c069070 con 0x7ff10c102ef0 2024-03-26T17:54:23.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.698+0000 7ff1027fc700 1 -- 172.21.15.115:0/400927573 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) ==== 137798+0+0 (secure 0 0 0) 0x7ff108029070 con 0x7ff10c102ef0 2024-03-26T17:54:23.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.699+0000 7ff1027fc700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0f80a2f20 0x7ff0f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:23.689 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.699+0000 7ff1027fc700 1 -- 172.21.15.115:0/400927573 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff1080ceaa0 con 0x7ff10c102ef0 2024-03-26T17:54:23.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.699+0000 7ff11097a700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0f80a2f20 0x7ff0f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:23.690 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.700+0000 7ff11097a700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0f80a2f20 0x7ff0f80a53e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009c60 tx=0x7ff0fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:23.691 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.703+0000 7ff1027fc700 1 -- 172.21.15.115:0/400927573 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff108030080 con 0x7ff10c102ef0 2024-03-26T17:54:23.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:23 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi049 2024-03-26T17:54:23.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:23 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:23 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:23 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:54:23.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:23 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi049 to 17088M 2024-03-26T17:54:23.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:23 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:23 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:23.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:23 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:23.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:23 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:23.896 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:23.908+0000 7ff1133df700 1 -- 172.21.15.115:0/400927573 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff10c0627f0 con 0x7ff0f80a2f20 2024-03-26T17:54:24.667 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:24 smithi115 ceph-mon[36174]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:24.667 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:24 smithi115 ceph-mon[36174]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T17:54:24.667 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:54:24.667 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:54:24.667 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:24.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:24 smithi049 ceph-mon[32331]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:24.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:24 smithi049 ceph-mon[32331]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T17:54:24.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:54:24.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:54:24.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:24.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:24 smithi171 ceph-mon[36033]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:24.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:24 smithi171 ceph-mon[36033]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T17:54:24.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:54:24.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:54:24.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:25.573 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:25 smithi115 ceph-mon[36174]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:54:25.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:25 smithi049 ceph-mon[32331]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:54:25.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:25 smithi171 ceph-mon[36033]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:54:26.681 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:26 smithi115 ceph-mon[36174]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:26.681 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:26 smithi115 ceph-mon[36174]: from='client.? 172.21.15.115:0/315198926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68c19f4c-7c26-4834-9bcb-d50b17b767e6"}]: dispatch 2024-03-26T17:54:26.682 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:26 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68c19f4c-7c26-4834-9bcb-d50b17b767e6"}]: dispatch 2024-03-26T17:54:26.682 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:26 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68c19f4c-7c26-4834-9bcb-d50b17b767e6"}]': finished 2024-03-26T17:54:26.682 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:26 smithi115 ceph-mon[36174]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T17:54:26.682 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:26.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:26 smithi049 ceph-mon[32331]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:26.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:26 smithi049 ceph-mon[32331]: from='client.? 172.21.15.115:0/315198926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68c19f4c-7c26-4834-9bcb-d50b17b767e6"}]: dispatch 2024-03-26T17:54:26.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:26 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68c19f4c-7c26-4834-9bcb-d50b17b767e6"}]: dispatch 2024-03-26T17:54:26.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:26 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68c19f4c-7c26-4834-9bcb-d50b17b767e6"}]': finished 2024-03-26T17:54:26.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:26 smithi049 ceph-mon[32331]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T17:54:26.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:26.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:26 smithi171 ceph-mon[36033]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:26.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:26 smithi171 ceph-mon[36033]: from='client.? 172.21.15.115:0/315198926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68c19f4c-7c26-4834-9bcb-d50b17b767e6"}]: dispatch 2024-03-26T17:54:26.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:26 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68c19f4c-7c26-4834-9bcb-d50b17b767e6"}]: dispatch 2024-03-26T17:54:26.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:26 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68c19f4c-7c26-4834-9bcb-d50b17b767e6"}]': finished 2024-03-26T17:54:26.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:26 smithi171 ceph-mon[36033]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T17:54:26.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:27.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:27 smithi115 ceph-mon[36174]: from='client.? 172.21.15.115:0/2452725616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:54:27.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:27 smithi049 ceph-mon[32331]: from='client.? 172.21.15.115:0/2452725616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:54:27.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:27 smithi171 ceph-mon[36033]: from='client.? 172.21.15.115:0/2452725616' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:54:28.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:28 smithi115 ceph-mon[36174]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:28.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:28 smithi049 ceph-mon[32331]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:28.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:28 smithi171 ceph-mon[36033]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:30.645 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:30 smithi115 ceph-mon[36174]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:30.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:30 smithi049 ceph-mon[32331]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:30.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:30 smithi171 ceph-mon[36033]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:32.461 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:32 smithi115 ceph-mon[36174]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:32.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:32 smithi049 ceph-mon[32331]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:32.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:32 smithi171 ceph-mon[36033]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:33.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:33 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T17:54:33.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:33 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:33.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T17:54:33.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:33.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T17:54:33.660 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:34.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:34 smithi115 ceph-mon[36174]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:34.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:34 smithi115 ceph-mon[36174]: Deploying daemon osd.1 on smithi115 2024-03-26T17:54:34.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:34 smithi049 ceph-mon[32331]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:34.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:34 smithi049 ceph-mon[32331]: Deploying daemon osd.1 on smithi115 2024-03-26T17:54:34.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:34 smithi171 ceph-mon[36033]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:34.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:34 smithi171 ceph-mon[36033]: Deploying daemon osd.1 on smithi115 2024-03-26T17:54:36.559 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:36 smithi115 ceph-mon[36174]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:36.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:36 smithi049 ceph-mon[32331]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:36.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:36 smithi171 ceph-mon[36033]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:38.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:38 smithi115 ceph-mon[36174]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:38.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:38 smithi049 ceph-mon[32331]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:38.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:38 smithi171 ceph-mon[36033]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:39.590 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:39 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:39.590 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:39 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:39.590 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:39 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:39.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:39 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:39.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:39 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:39.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:39 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:39.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:39 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:39.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:39 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:39.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:39 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:40.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:40 smithi115 ceph-mon[36174]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:40.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:40 smithi049 ceph-mon[32331]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:40.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:40 smithi171 ceph-mon[36033]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.272+0000 7ff1027fc700 1 -- 172.21.15.115:0/400927573 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff10c0627f0 con 0x7ff0f80a2f20 2024-03-26T17:54:42.261 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 1 on host 'smithi115' 2024-03-26T17:54:42.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.274+0000 7ff1133df700 1 -- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0f80a2f20 msgr2=0x7ff0f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:42.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.274+0000 7ff1133df700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0f80a2f20 0x7ff0f80a53e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009c60 tx=0x7ff0fc009260 comp rx=0 tx=0).stop 2024-03-26T17:54:42.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.274+0000 7ff1133df700 1 -- 172.21.15.115:0/400927573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff10c102ef0 msgr2=0x7ff10c102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:42.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.274+0000 7ff1133df700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff10c102ef0 0x7ff10c102850 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff108005910 tx=0x7ff10800bbb0 comp rx=0 tx=0).stop 2024-03-26T17:54:42.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.274+0000 7ff1133df700 1 -- 172.21.15.115:0/400927573 shutdown_connections 2024-03-26T17:54:42.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.274+0000 7ff1133df700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0f80a2f20 0x7ff0f80a53e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:42.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.275+0000 7ff1133df700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff10c108150 0x7ff10c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:42.263 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.275+0000 7ff1133df700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff10c102ef0 0x7ff10c102850 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:42.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.275+0000 7ff1133df700 1 --2- 172.21.15.115:0/400927573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c105820 0x7ff10c100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:42.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.275+0000 7ff1133df700 1 -- 172.21.15.115:0/400927573 >> 172.21.15.115:0/400927573 conn(0x7ff10c0fc8a0 msgr2=0x7ff10c103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:42.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.275+0000 7ff1133df700 1 -- 172.21.15.115:0/400927573 shutdown_connections 2024-03-26T17:54:42.264 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:54:42.275+0000 7ff1133df700 1 -- 172.21.15.115:0/400927573 wait complete. 2024-03-26T17:54:42.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:42 smithi115 ceph-mon[36174]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:42.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:42.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:42.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:42.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:42 smithi049 ceph-mon[32331]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:42.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:42.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:42.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:42 smithi171 ceph-mon[36033]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:42.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:42.922 DEBUG:teuthology.orchestra.run.smithi115:osd.1> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.1.service 2024-03-26T17:54:42.924 INFO:tasks.cephadm:Deploying osd.2 on smithi171 with /dev/vg_nvme/lv_4... 2024-03-26T17:54:42.925 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T17:54:43.171 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:54:43.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:43 smithi115 ceph-mon[36174]: from='osd.1 [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:54:43.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:43 smithi115 ceph-mon[36174]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:54:43.589 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:43 smithi171 ceph-mon[36033]: from='osd.1 [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:54:43.589 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:43 smithi171 ceph-mon[36033]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:54:43.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:43 smithi049 ceph-mon[32331]: from='osd.1 [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:54:43.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:43 smithi049 ceph-mon[32331]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:54:44.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:44 smithi171 ceph-mon[36033]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:44.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:44 smithi171 ceph-mon[36033]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T17:54:44.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:44 smithi171 ceph-mon[36033]: from='osd.1 [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:54:44.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:44 smithi171 ceph-mon[36033]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T17:54:44.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:44.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:44 smithi171 ceph-mon[36033]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:54:44.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:44 smithi115 ceph-mon[36174]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:44.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:44 smithi115 ceph-mon[36174]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T17:54:44.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:44 smithi115 ceph-mon[36174]: from='osd.1 [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:54:44.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:44 smithi115 ceph-mon[36174]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T17:54:44.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:44.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:44 smithi115 ceph-mon[36174]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:54:44.743 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 17:54:44 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1[40773]: 2024-03-26T17:54:44.428+0000 7ff35d4ae700 -1 osd.1 0 waiting for initial osdmap 2024-03-26T17:54:44.744 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 17:54:44 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1[40773]: 2024-03-26T17:54:44.431+0000 7ff3592a1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:54:44.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:44 smithi049 ceph-mon[32331]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:44.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:44 smithi049 ceph-mon[32331]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T17:54:44.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:44 smithi049 ceph-mon[32331]: from='osd.1 [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:54:44.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:44 smithi049 ceph-mon[32331]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T17:54:44.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:44.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:44 smithi049 ceph-mon[32331]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:54:44.876 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T17:54:44.876 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T17:54:44.877 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-03-26T17:54:44.877 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-03-26T17:54:44.877 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112888 s, 929 MB/s 2024-03-26T17:54:44.877 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-03-26T17:54:45.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:45 smithi171 ceph-mon[36033]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:54:45.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:45 smithi171 ceph-mon[36033]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T17:54:45.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:45.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:45.460 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi171:vg_nvme/lv_4 2024-03-26T17:54:45.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:45 smithi115 ceph-mon[36174]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:54:45.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:45 smithi115 ceph-mon[36174]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T17:54:45.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:45.493 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:45.706 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:54:45.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:45 smithi049 ceph-mon[32331]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:54:45.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:45 smithi049 ceph-mon[32331]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T17:54:45.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:45.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:46.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:54:46.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:54:46.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:46.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: osd.1 [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] boot 2024-03-26T17:54:46.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T17:54:46.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:46.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.684 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:54:46.685 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.685 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:46.685 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:46.685 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:54:46.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:54:46.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:46.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: osd.1 [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] boot 2024-03-26T17:54:46.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T17:54:46.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:46.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:54:46.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:46.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:46.744 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: osd.1 [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] boot 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:46.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:54:47.270 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.279+0000 7f43debbb700 1 -- 172.21.15.171:0/1226011186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d81057d0 msgr2=0x7f43d8107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:47.270 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.279+0000 7f43debbb700 1 --2- 172.21.15.171:0/1226011186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d81057d0 0x7f43d8107bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009990 tx=0x7f43c8009ca0 comp rx=0 tx=0).stop 2024-03-26T17:54:47.270 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.279+0000 7f43debbb700 1 -- 172.21.15.171:0/1226011186 shutdown_connections 2024-03-26T17:54:47.270 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.279+0000 7f43debbb700 1 --2- 172.21.15.171:0/1226011186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43d8108100 0x7f43d810a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:47.270 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.279+0000 7f43debbb700 1 --2- 172.21.15.171:0/1226011186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d8102ea0 0x7f43d8105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:47.270 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.279+0000 7f43debbb700 1 --2- 172.21.15.171:0/1226011186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d81057d0 0x7f43d8107bc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:47.270 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.279+0000 7f43debbb700 1 -- 172.21.15.171:0/1226011186 >> 172.21.15.171:0/1226011186 conn(0x7f43d80fc8d0 msgr2=0x7f43d80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:47.270 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.279+0000 7f43debbb700 1 -- 172.21.15.171:0/1226011186 shutdown_connections 2024-03-26T17:54:47.270 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.279+0000 7f43debbb700 1 -- 172.21.15.171:0/1226011186 wait complete. 2024-03-26T17:54:47.270 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.279+0000 7f43debbb700 1 Processor -- start 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43debbb700 1 -- start start 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43debbb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d8102ea0 0x7f43d8102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43debbb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43d81057d0 0x7f43d8100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43debbb700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d8108100 0x7f43d8101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43debbb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f43d810e020 con 0x7f43d8102ea0 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43debbb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f43d810dea0 con 0x7f43d81057d0 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43debbb700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f43d810e1a0 con 0x7f43d8108100 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43dd158700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d8108100 0x7f43d8101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43dd158700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d8108100 0x7f43d8101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:49246/0 (socket says 172.21.15.171:49246) 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43dd158700 1 -- 172.21.15.171:0/1151271757 learned_addr learned my addr 172.21.15.171:0/1151271757 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43cffff700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43d81057d0 0x7f43d8100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43dc957700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d8102ea0 0x7f43d8102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:47.271 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43dd158700 1 -- 172.21.15.171:0/1151271757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43d81057d0 msgr2=0x7f43d8100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:47.272 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43dd158700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43d81057d0 0x7f43d8100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:47.272 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43dd158700 1 -- 172.21.15.171:0/1151271757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d8102ea0 msgr2=0x7f43d8102800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:47.272 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43dd158700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d8102ea0 0x7f43d8102800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:47.272 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43dd158700 1 -- 172.21.15.171:0/1151271757 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d8101c40 con 0x7f43d8108100 2024-03-26T17:54:47.272 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43dd158700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d8108100 0x7f43d8101390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f43d000ada0 tx=0x7f43d000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:47.272 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.280+0000 7f43cdffb700 1 -- 172.21.15.171:0/1151271757 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0017020 con 0x7f43d8108100 2024-03-26T17:54:47.272 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.281+0000 7f43cdffb700 1 -- 172.21.15.171:0/1151271757 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d000cdb0 con 0x7f43d8108100 2024-03-26T17:54:47.272 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.281+0000 7f43cdffb700 1 -- 172.21.15.171:0/1151271757 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d00053a0 con 0x7f43d8108100 2024-03-26T17:54:47.272 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.281+0000 7f43debbb700 1 -- 172.21.15.171:0/1151271757 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d81a27a0 con 0x7f43d8108100 2024-03-26T17:54:47.272 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.281+0000 7f43debbb700 1 -- 172.21.15.171:0/1151271757 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d81a2cb0 con 0x7f43d8108100 2024-03-26T17:54:47.273 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.282+0000 7f43debbb700 1 -- 172.21.15.171:0/1151271757 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d8069070 con 0x7f43d8108100 2024-03-26T17:54:47.273 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.283+0000 7f43cdffb700 1 -- 172.21.15.171:0/1151271757 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 15) ==== 137798+0+0 (secure 0 0 0) 0x7f43d000c8d0 con 0x7f43d8108100 2024-03-26T17:54:47.273 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.283+0000 7f43cdffb700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f43c00a2f20 0x7f43c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:47.273 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.283+0000 7f43cdffb700 1 -- 172.21.15.171:0/1151271757 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f43d00cd130 con 0x7f43d8108100 2024-03-26T17:54:47.273 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.283+0000 7f43dc957700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f43c00a2f20 0x7f43c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:47.273 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.284+0000 7f43dc957700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f43c00a2f20 0x7f43c00a53e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43d4007c60 tx=0x7f43d40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:47.277 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.288+0000 7f43cdffb700 1 -- 172.21.15.171:0/1151271757 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f43d0021020 con 0x7f43d8108100 2024-03-26T17:54:47.485 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:54:47.496+0000 7f43debbb700 1 -- 172.21.15.171:0/1151271757 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f43d80627f0 con 0x7f43c00a2f20 2024-03-26T17:54:47.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:47 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi115 2024-03-26T17:54:47.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:47 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi115 to 17080M 2024-03-26T17:54:47.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:47 smithi171 ceph-mon[36033]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T17:54:47.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:47 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi115 2024-03-26T17:54:47.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:47 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi115 to 17080M 2024-03-26T17:54:47.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:47 smithi115 ceph-mon[36174]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T17:54:47.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:47 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi115 2024-03-26T17:54:47.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:47 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi115 to 17080M 2024-03-26T17:54:47.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:47 smithi049 ceph-mon[32331]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T17:54:48.745 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:48 smithi115 ceph-mon[36174]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:48.746 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:48 smithi115 ceph-mon[36174]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:54:48.746 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:48 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:54:48.746 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:48 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:54:48.746 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:48 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:48.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:48 smithi049 ceph-mon[32331]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:48.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:48 smithi049 ceph-mon[32331]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:54:48.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:48 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:54:48.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:48 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:54:48.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:48 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:48.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:48 smithi171 ceph-mon[36033]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:48.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:48 smithi171 ceph-mon[36033]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:54:48.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:48 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:54:48.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:48 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:54:48.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:48 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:50.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:50 smithi115 ceph-mon[36174]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:50.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:50 smithi115 ceph-mon[36174]: from='client.? 172.21.15.171:0/2827412373' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01428517-2a3a-43fa-88ce-fb0b460d78eb"}]: dispatch 2024-03-26T17:54:50.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:50 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01428517-2a3a-43fa-88ce-fb0b460d78eb"}]: dispatch 2024-03-26T17:54:50.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:50 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01428517-2a3a-43fa-88ce-fb0b460d78eb"}]': finished 2024-03-26T17:54:50.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:50 smithi115 ceph-mon[36174]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T17:54:50.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:50 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:54:50.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:50 smithi049 ceph-mon[32331]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:50.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:50 smithi049 ceph-mon[32331]: from='client.? 172.21.15.171:0/2827412373' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01428517-2a3a-43fa-88ce-fb0b460d78eb"}]: dispatch 2024-03-26T17:54:50.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:50 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01428517-2a3a-43fa-88ce-fb0b460d78eb"}]: dispatch 2024-03-26T17:54:50.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:50 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01428517-2a3a-43fa-88ce-fb0b460d78eb"}]': finished 2024-03-26T17:54:50.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:50 smithi049 ceph-mon[32331]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T17:54:50.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:50 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:54:50.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:50 smithi171 ceph-mon[36033]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:50.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:50 smithi171 ceph-mon[36033]: from='client.? 172.21.15.171:0/2827412373' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01428517-2a3a-43fa-88ce-fb0b460d78eb"}]: dispatch 2024-03-26T17:54:50.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:50 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01428517-2a3a-43fa-88ce-fb0b460d78eb"}]: dispatch 2024-03-26T17:54:50.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:50 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01428517-2a3a-43fa-88ce-fb0b460d78eb"}]': finished 2024-03-26T17:54:50.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:50 smithi171 ceph-mon[36033]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T17:54:50.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:50 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:54:51.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:51 smithi115 ceph-mon[36174]: from='client.? 172.21.15.171:0/3830469541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:54:51.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:51 smithi049 ceph-mon[32331]: from='client.? 172.21.15.171:0/3830469541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:54:51.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:51 smithi171 ceph-mon[36033]: from='client.? 172.21.15.171:0/3830469541' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:54:52.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:52 smithi115 ceph-mon[36174]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:52.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:52 smithi049 ceph-mon[32331]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:52.762 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:52 smithi171 ceph-mon[36033]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:54.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:54 smithi171 ceph-mon[36033]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:54.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:54 smithi115 ceph-mon[36174]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:54.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:54 smithi049 ceph-mon[32331]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:56.715 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:56 smithi171 ceph-mon[36033]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:56.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:56 smithi115 ceph-mon[36174]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:56.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:56 smithi049 ceph-mon[32331]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:57.701 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T17:54:57.702 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:57.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T17:54:57.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:57.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T17:54:57.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:58.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:58 smithi171 ceph-mon[36033]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:58.704 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:54:58 smithi171 ceph-mon[36033]: Deploying daemon osd.2 on smithi171 2024-03-26T17:54:58.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:58 smithi115 ceph-mon[36174]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:58.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:54:58 smithi115 ceph-mon[36174]: Deploying daemon osd.2 on smithi171 2024-03-26T17:54:58.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:58 smithi049 ceph-mon[32331]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:54:58.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:54:58 smithi049 ceph-mon[32331]: Deploying daemon osd.2 on smithi171 2024-03-26T17:55:00.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:00 smithi115 ceph-mon[36174]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:00.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:00 smithi049 ceph-mon[32331]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:00.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:00 smithi171 ceph-mon[36033]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:02.733 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:02 smithi171 ceph-mon[36033]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:02.733 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:02 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:02.734 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:02 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:02.734 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:02 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:02.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:02 smithi115 ceph-mon[36174]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:02.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:02 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:02.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:02 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:02.743 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:02 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:02.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:02 smithi049 ceph-mon[32331]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:02.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:02 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:02.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:02 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:02.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:02 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:04.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:04 smithi115 ceph-mon[36174]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:04.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:04 smithi049 ceph-mon[32331]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:04.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:04 smithi171 ceph-mon[36033]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:05.655 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.666+0000 7f43cdffb700 1 -- 172.21.15.171:0/1151271757 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f43d80627f0 con 0x7f43c00a2f20 2024-03-26T17:55:05.655 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 2 on host 'smithi171' 2024-03-26T17:55:05.656 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 -- 172.21.15.171:0/1151271757 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f43c00a2f20 msgr2=0x7f43c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:05.656 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f43c00a2f20 0x7f43c00a53e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43d4007c60 tx=0x7f43d40073d0 comp rx=0 tx=0).stop 2024-03-26T17:55:05.657 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 -- 172.21.15.171:0/1151271757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d8108100 msgr2=0x7f43d8101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:05.657 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d8108100 0x7f43d8101390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f43d000ada0 tx=0x7f43d000c330 comp rx=0 tx=0).stop 2024-03-26T17:55:05.657 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 -- 172.21.15.171:0/1151271757 shutdown_connections 2024-03-26T17:55:05.657 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f43c00a2f20 0x7f43c00a53e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:05.657 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43d8108100 0x7f43d8101390 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:05.657 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f43d8102ea0 0x7f43d8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:05.657 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 --2- 172.21.15.171:0/1151271757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43d81057d0 0x7f43d8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:05.657 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 -- 172.21.15.171:0/1151271757 >> 172.21.15.171:0/1151271757 conn(0x7f43d80fc8d0 msgr2=0x7f43d81037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:05.657 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.668+0000 7f43debbb700 1 -- 172.21.15.171:0/1151271757 shutdown_connections 2024-03-26T17:55:05.657 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:55:05.669+0000 7f43debbb700 1 -- 172.21.15.171:0/1151271757 wait complete. 2024-03-26T17:55:05.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:05 smithi171 ceph-mon[36033]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:05.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:05.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:05.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:05.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:05.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:05.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:05.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:05.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:05 smithi049 ceph-mon[32331]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:06.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:06.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:06.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:06.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.035 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:05 smithi115 ceph-mon[36174]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:06.035 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.035 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.035 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:06.036 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:06.036 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.036 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.036 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:06.036 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:06.259 DEBUG:teuthology.orchestra.run.smithi171:osd.2> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.2.service 2024-03-26T17:55:06.261 INFO:tasks.cephadm:Deploying osd.3 on smithi049 with /dev/vg_nvme/lv_3... 2024-03-26T17:55:06.262 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T17:55:06.519 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:55:06.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:06 smithi171 ceph-mon[36033]: from='osd.2 [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:55:06.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:06 smithi171 ceph-mon[36033]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:55:06.945 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:06 smithi049 ceph-mon[32331]: from='osd.2 [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:55:06.945 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:06 smithi049 ceph-mon[32331]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:55:07.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:06 smithi115 ceph-mon[36174]: from='osd.2 [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:55:07.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:06 smithi115 ceph-mon[36174]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:55:07.912 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:07 smithi049 ceph-mon[32331]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:07.912 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:07 smithi049 ceph-mon[32331]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T17:55:07.912 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:07 smithi049 ceph-mon[32331]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T17:55:07.912 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:07 smithi049 ceph-mon[32331]: from='osd.2 [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:55:07.912 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:07 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:07.912 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:07 smithi049 ceph-mon[32331]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:55:08.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:07 smithi171 ceph-mon[36033]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:08.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:07 smithi171 ceph-mon[36033]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T17:55:08.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:07 smithi171 ceph-mon[36033]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T17:55:08.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:07 smithi171 ceph-mon[36033]: from='osd.2 [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:55:08.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:07 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:08.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:07 smithi171 ceph-mon[36033]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:55:08.112 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 17:55:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2[41186]: 2024-03-26T17:55:07.775+0000 7f5560403700 -1 osd.2 0 waiting for initial osdmap 2024-03-26T17:55:08.112 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 17:55:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2[41186]: 2024-03-26T17:55:07.779+0000 7f555c1f6700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:55:08.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:07 smithi115 ceph-mon[36174]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:08.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:07 smithi115 ceph-mon[36174]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T17:55:08.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:07 smithi115 ceph-mon[36174]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T17:55:08.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:07 smithi115 ceph-mon[36174]: from='osd.2 [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:55:08.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:07 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:08.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:07 smithi115 ceph-mon[36174]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:55:08.644 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T17:55:08.644 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T17:55:08.644 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-26T17:55:08.644 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-26T17:55:08.644 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00863744 s, 1.2 GB/s 2024-03-26T17:55:08.644 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-26T17:55:08.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:08 smithi171 ceph-mon[36033]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:55:08.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:08 smithi171 ceph-mon[36033]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T17:55:08.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:08 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:08.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:08 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:08.961 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:08 smithi049 ceph-mon[32331]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:55:08.961 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:08 smithi049 ceph-mon[32331]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T17:55:08.961 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:08 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:08.961 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:08 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:09.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:08 smithi115 ceph-mon[36174]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:55:09.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:08 smithi115 ceph-mon[36174]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T17:55:09.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:08 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:09.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:08 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:09.395 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_3 2024-03-26T17:55:09.646 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:55:10.035 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: osd.2 [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] boot 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi171 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi171 to 17080M 2024-03-26T17:55:10.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:10.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:10.037 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:55:10.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: osd.2 [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] boot 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi171 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi171 to 17080M 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:10.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:10.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:55:10.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:55:10.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:55:10.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: osd.2 [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] boot 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi171 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi171 to 17080M 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:10.243 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:11.216 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:11 smithi049 ceph-mon[32331]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T17:55:11.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.470+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/191521121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4102ef0 msgr2=0x7fe6a41097a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:11.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.470+0000 7fe6ab1d9700 1 --2- 172.21.15.49:0/191521121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4102ef0 0x7fe6a41097a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00b0d0 tx=0x7fe69c00b3e0 comp rx=0 tx=0).stop 2024-03-26T17:55:11.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.470+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/191521121 shutdown_connections 2024-03-26T17:55:11.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.470+0000 7fe6ab1d9700 1 --2- 172.21.15.49:0/191521121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4102ef0 0x7fe6a41097a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:11.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.470+0000 7fe6ab1d9700 1 --2- 172.21.15.49:0/191521121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a4101350 0x7fe6a4101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:11.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.470+0000 7fe6ab1d9700 1 --2- 172.21.15.49:0/191521121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a4102550 0x7fe6a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:11.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.470+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/191521121 >> 172.21.15.49:0/191521121 conn(0x7fe6a40fc910 msgr2=0x7fe6a40fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:11.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.470+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/191521121 shutdown_connections 2024-03-26T17:55:11.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.470+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/191521121 wait complete. 2024-03-26T17:55:11.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6ab1d9700 1 Processor -- start 2024-03-26T17:55:11.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6ab1d9700 1 -- start start 2024-03-26T17:55:11.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6ab1d9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a4101350 0x7fe6a4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:11.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6ab1d9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4102550 0x7fe6a4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6ab1d9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a4102ef0 0x7fe6a419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6ab1d9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a410cea0 con 0x7fe6a4102550 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6ab1d9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a410cd20 con 0x7fe6a4101350 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6ab1d9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a410d020 con 0x7fe6a4102ef0 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6a3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4102550 0x7fe6a4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6a3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4102550 0x7fe6a4198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42862/0 (socket says 172.21.15.49:42862) 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6a3fff700 1 -- 172.21.15.49:0/3355083149 learned_addr learned my addr 172.21.15.49:0/3355083149 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.471+0000 7fe6a8f75700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a4101350 0x7fe6a4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6a3fff700 1 -- 172.21.15.49:0/3355083149 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a4102ef0 msgr2=0x7fe6a419fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6a3fff700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a4102ef0 0x7fe6a419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6a3fff700 1 -- 172.21.15.49:0/3355083149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a4101350 msgr2=0x7fe6a4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6a3fff700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a4101350 0x7fe6a4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:11.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6a3fff700 1 -- 172.21.15.49:0/3355083149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a41a0180 con 0x7fe6a4102550 2024-03-26T17:55:11.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6a3fff700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4102550 0x7fe6a4198a60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe69000d9b0 tx=0x7fe69000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:11.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6a1ffb700 1 -- 172.21.15.49:0/3355083149 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690016070 con 0x7fe6a4102550 2024-03-26T17:55:11.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/3355083149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a41a0470 con 0x7fe6a4102550 2024-03-26T17:55:11.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6a1ffb700 1 -- 172.21.15.49:0/3355083149 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe690004d10 con 0x7fe6a4102550 2024-03-26T17:55:11.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6a1ffb700 1 -- 172.21.15.49:0/3355083149 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690005630 con 0x7fe6a4102550 2024-03-26T17:55:11.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.472+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/3355083149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a41a09b0 con 0x7fe6a4102550 2024-03-26T17:55:11.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.473+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/3355083149 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a4069070 con 0x7fe6a4102550 2024-03-26T17:55:11.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.474+0000 7fe6a1ffb700 1 -- 172.21.15.49:0/3355083149 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) ==== 137798+0+0 (secure 0 0 0) 0x7fe690004750 con 0x7fe6a4102550 2024-03-26T17:55:11.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.475+0000 7fe6a1ffb700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe6940a2f20 0x7fe6940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:11.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.475+0000 7fe6a1ffb700 1 -- 172.21.15.49:0/3355083149 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7fe6900ce240 con 0x7fe6a4102550 2024-03-26T17:55:11.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.475+0000 7fe6a8f75700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe6940a2f20 0x7fe6940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:11.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.476+0000 7fe6a8f75700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe6940a2f20 0x7fe6940a53e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6a41023b0 tx=0x7fe698009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:11.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.479+0000 7fe6a1ffb700 1 -- 172.21.15.49:0/3355083149 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe69001f020 con 0x7fe6a4102550 2024-03-26T17:55:11.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:11 smithi115 ceph-mon[36174]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T17:55:11.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:11 smithi171 ceph-mon[36033]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T17:55:11.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:11.694+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/3355083149 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe6a40fea00 con 0x7fe6940a2f20 2024-03-26T17:55:12.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:12 smithi049 ceph-mon[32331]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:12.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:55:12.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T17:55:12.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:12 smithi049 ceph-mon[32331]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T17:55:12.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:55:12.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:55:12.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:55:12.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:12.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:12 smithi115 ceph-mon[36174]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:12.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:55:12.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T17:55:12.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:12 smithi115 ceph-mon[36174]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T17:55:12.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:55:12.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:55:12.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:55:12.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:12.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:12 smithi171 ceph-mon[36033]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:12.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:55:12.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T17:55:12.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:12 smithi171 ceph-mon[36033]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T17:55:12.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:55:12.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:55:12.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:55:12.612 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:13.007 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 17:55:12 smithi049 sudo[51123]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:55:13.007 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 17:55:12 smithi049 sudo[51123]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:13.007 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 17:55:12 smithi049 sudo[51123]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:13.007 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 17:55:12 smithi049 sudo[51123]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:13.465 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:13 smithi115 ceph-mon[36174]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:55:13.465 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T17:55:13.465 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:13 smithi115 ceph-mon[36174]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T17:55:13.493 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:13 smithi049 ceph-mon[32331]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:55:13.493 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T17:55:13.493 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:13 smithi049 ceph-mon[32331]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T17:55:13.494 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 17:55:13 smithi049 sudo[51130]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:55:13.494 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 17:55:13 smithi049 sudo[51130]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:13.494 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 17:55:13 smithi049 sudo[51130]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:13.494 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 17:55:13 smithi049 sudo[51130]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:13.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:13 smithi171 ceph-mon[36033]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:55:13.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T17:55:13.611 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:13 smithi171 ceph-mon[36033]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T17:55:13.719 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 17:55:13 smithi115 sudo[44478]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:55:13.719 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 17:55:13 smithi115 sudo[44478]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:13.720 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 17:55:13 smithi115 sudo[44478]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:13.720 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 17:55:13 smithi115 sudo[44478]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:13.992 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 17:55:13 smithi115 sudo[44481]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:55:13.992 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 17:55:13 smithi115 sudo[44481]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:13.992 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 17:55:13 smithi115 sudo[44481]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:13.992 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 17:55:13 smithi115 sudo[44481]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:14.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:14.242+0000 7fe6a1ffb700 1 -- 172.21.15.49:0/3355083149 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fe6900984a0 con 0x7fe6a4102550 2024-03-26T17:55:14.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:14 smithi171 ceph-mon[36033]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:14.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:14 smithi171 ceph-mon[36033]: osdmap e22: 3 total, 3 up, 3 in 2024-03-26T17:55:14.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:14 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:14.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:14 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:14.243 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 17:55:14 smithi171 sudo[44815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:55:14.243 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 17:55:14 smithi171 sudo[44815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:14.243 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 17:55:14 smithi171 sudo[44815]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:14.243 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 17:55:14 smithi171 sudo[44815]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:14.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:14 smithi115 ceph-mon[36174]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:14.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:14 smithi115 ceph-mon[36174]: osdmap e22: 3 total, 3 up, 3 in 2024-03-26T17:55:14.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:14 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:14.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:14 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:14.507 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:14 smithi049 ceph-mon[32331]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:14.507 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:14 smithi049 ceph-mon[32331]: osdmap e22: 3 total, 3 up, 3 in 2024-03-26T17:55:14.507 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:14 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:14.507 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:14 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:14.611 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 17:55:14 smithi171 sudo[44818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:55:14.611 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 17:55:14 smithi171 sudo[44818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:14.611 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 17:55:14 smithi171 sudo[44818]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:14.611 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 17:55:14 smithi171 sudo[44818]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:15.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:14 smithi049 sudo[51205]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:55:15.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:14 smithi049 sudo[51205]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:15.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:14 smithi049 sudo[51205]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:15.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:14 smithi049 sudo[51205]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:15.507 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 sudo[51208]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:55:15.507 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 sudo[51208]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:15.507 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 sudo[51208]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:15.507 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 sudo[51208]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:15.665 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 sudo[44821]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:55:15.666 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 sudo[44821]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:15.666 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 sudo[44821]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:15.666 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 sudo[44821]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:15.906 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T17:55:15.906 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:15.906 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:55:15.906 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2427345859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b332db-6ccf-45ea-ae70-bd6471ce4e5e"}]: dispatch 2024-03-26T17:55:15.906 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:55:15.906 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b332db-6ccf-45ea-ae70-bd6471ce4e5e"}]: dispatch 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1b332db-6ccf-45ea-ae70-bd6471ce4e5e"}]': finished 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 sudo[44484]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 sudo[44484]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:15.907 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 sudo[44484]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:16.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T17:55:16.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:16.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:55:16.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2427345859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b332db-6ccf-45ea-ae70-bd6471ce4e5e"}]: dispatch 2024-03-26T17:55:16.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:55:16.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:55:16.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b332db-6ccf-45ea-ae70-bd6471ce4e5e"}]: dispatch 2024-03-26T17:55:16.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:55:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:55:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1b332db-6ccf-45ea-ae70-bd6471ce4e5e"}]': finished 2024-03-26T17:55:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T17:55:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:55:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:55:16.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:55:16.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 sudo[44824]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:55:16.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 sudo[44824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 sudo[44824]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 sudo[44824]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2427345859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b332db-6ccf-45ea-ae70-bd6471ce4e5e"}]: dispatch 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b332db-6ccf-45ea-ae70-bd6471ce4e5e"}]: dispatch 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1b332db-6ccf-45ea-ae70-bd6471ce4e5e"}]': finished 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T17:55:16.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:16.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:55:16.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:55:16.113 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:55:16.161 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:15 smithi115 sudo[44484]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:16.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 sudo[44487]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:55:16.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 sudo[44487]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:55:16.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 sudo[44487]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:55:16.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 sudo[44487]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:55:16.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 ceph-mon[36174]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:16.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 ceph-mon[36174]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:55:16.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 ceph-mon[36174]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:55:16.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 ceph-mon[36174]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:55:16.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:55:16.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:55:16.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:55:16.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/455456837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:55:16.993 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:16 smithi115 ceph-mon[36174]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:55:17.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:16 smithi049 ceph-mon[32331]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:17.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:16 smithi049 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:55:17.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:16 smithi049 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:55:17.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:16 smithi049 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:55:17.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:16 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:55:17.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:16 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:55:17.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:16 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:55:17.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:16 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/455456837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:55:17.008 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:16 smithi049 ceph-mon[32331]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:55:17.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:16 smithi171 ceph-mon[36033]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:17.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:16 smithi171 ceph-mon[36033]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:55:17.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:16 smithi171 ceph-mon[36033]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:55:17.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:16 smithi171 ceph-mon[36033]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:55:17.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:16 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:55:17.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:16 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:55:17.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:16 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:55:17.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:16 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/455456837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:55:17.112 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:16 smithi171 ceph-mon[36033]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:55:17.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:17 smithi115 ceph-mon[36174]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:18.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:17 smithi049 ceph-mon[32331]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:18.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:17 smithi171 ceph-mon[36033]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:19.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:19 smithi115 ceph-mon[36174]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:20.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:19 smithi049 ceph-mon[32331]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:20.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:19 smithi171 ceph-mon[36033]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:21.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:21 smithi115 ceph-mon[36174]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:22.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:21 smithi049 ceph-mon[32331]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:22.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:21 smithi171 ceph-mon[36033]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:23.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:23 smithi049 ceph-mon[32331]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:23.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:23 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T17:55:23.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:23 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:23.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:23 smithi049 ceph-mon[32331]: Deploying daemon osd.3 on smithi049 2024-03-26T17:55:23.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:23 smithi115 ceph-mon[36174]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:23.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:23 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T17:55:23.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:23 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:23.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:23 smithi115 ceph-mon[36174]: Deploying daemon osd.3 on smithi049 2024-03-26T17:55:24.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:23 smithi171 ceph-mon[36033]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:24.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:23 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T17:55:24.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:23 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:24.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:23 smithi171 ceph-mon[36033]: Deploying daemon osd.3 on smithi049 2024-03-26T17:55:25.939 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:25 smithi049 ceph-mon[32331]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:25.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:25 smithi115 ceph-mon[36174]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:26.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:25 smithi171 ceph-mon[36033]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:27.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:27 smithi115 ceph-mon[36174]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:28.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:27 smithi049 ceph-mon[32331]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:28.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:27 smithi171 ceph-mon[36033]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:30.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:29 smithi049 ceph-mon[32331]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:30.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:29 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:30.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:29 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:30.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:29 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:30.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:29 smithi171 ceph-mon[36033]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:30.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:29 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:30.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:29 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:30.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:29 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:30.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:29 smithi115 ceph-mon[36174]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:30.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:29 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:30.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:29 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:30.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:29 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:32.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:31 smithi049 ceph-mon[32331]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:32.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:31 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:32.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:31 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:32.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:31 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:32.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:31 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:32.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:31 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:32.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:31 smithi171 ceph-mon[36033]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:32.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:31 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:32.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:31 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:32.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:31 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:32.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:31 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:32.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:31 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:32.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:31 smithi115 ceph-mon[36174]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:32.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:31 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:32.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:31 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:32.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:31 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:32.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:31 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:32.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:31 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:32.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.605+0000 7fe6a1ffb700 1 -- 172.21.15.49:0/3355083149 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe6a40fea00 con 0x7fe6940a2f20 2024-03-26T17:55:32.593 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 3 on host 'smithi049' 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/3355083149 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe6940a2f20 msgr2=0x7fe6940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe6940a2f20 0x7fe6940a53e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6a41023b0 tx=0x7fe698009260 comp rx=0 tx=0).stop 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/3355083149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4102550 msgr2=0x7fe6a4198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4102550 0x7fe6a4198a60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe69000d9b0 tx=0x7fe69000dcc0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/3355083149 shutdown_connections 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe6940a2f20 0x7fe6940a53e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a4102ef0 0x7fe6a419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6a4101350 0x7fe6a4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 --2- 172.21.15.49:0/3355083149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4102550 0x7fe6a4198a60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/3355083149 >> 172.21.15.49:0/3355083149 conn(0x7fe6a40fc910 msgr2=0x7fe6a40fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/3355083149 shutdown_connections 2024-03-26T17:55:32.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:55:32.608+0000 7fe6ab1d9700 1 -- 172.21.15.49:0/3355083149 wait complete. 2024-03-26T17:55:33.282 DEBUG:teuthology.orchestra.run.smithi049:osd.3> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.3.service 2024-03-26T17:55:33.285 INFO:tasks.cephadm:Deploying osd.4 on smithi115 with /dev/vg_nvme/lv_3... 2024-03-26T17:55:33.285 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T17:55:33.529 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:55:33.824 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:33 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:33.824 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:33 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:33.824 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:33 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:33.824 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:33 smithi115 ceph-mon[36174]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:33.825 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:33 smithi115 ceph-mon[36174]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T17:55:33.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:33.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:33.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:33.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:33 smithi171 ceph-mon[36033]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:33.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:33 smithi171 ceph-mon[36033]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T17:55:34.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:34.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:34.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:34.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:33 smithi049 ceph-mon[32331]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:34.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:33 smithi049 ceph-mon[32331]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T17:55:34.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:34 smithi049 ceph-mon[32331]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T17:55:34.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:34 smithi049 ceph-mon[32331]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T17:55:34.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:34 smithi049 ceph-mon[32331]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:55:34.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:34 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:34.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:34 smithi171 ceph-mon[36033]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T17:55:34.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:34 smithi171 ceph-mon[36033]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T17:55:34.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:34 smithi171 ceph-mon[36033]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:55:34.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:34 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:34.940 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:34 smithi115 ceph-mon[36174]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T17:55:34.941 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:34 smithi115 ceph-mon[36174]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T17:55:34.941 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:34 smithi115 ceph-mon[36174]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:55:34.941 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:34 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:35.257 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 17:55:35 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T17:55:35.052+0000 7f009dd6f700 -1 osd.3 0 waiting for initial osdmap 2024-03-26T17:55:35.258 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 17:55:35 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T17:55:35.057+0000 7f0095359700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:55:35.623 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T17:55:35.623 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T17:55:35.623 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-03-26T17:55:35.623 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-03-26T17:55:35.623 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130609 s, 803 MB/s 2024-03-26T17:55:35.623 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-03-26T17:55:35.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:35 smithi171 ceph-mon[36033]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:55:35.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:35 smithi171 ceph-mon[36033]: osdmap e25: 4 total, 3 up, 4 in 2024-03-26T17:55:35.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:35.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:35.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:35 smithi171 ceph-mon[36033]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:35.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:35 smithi171 ceph-mon[36033]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' 2024-03-26T17:55:35.956 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:35 smithi115 ceph-mon[36174]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:55:35.956 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:35 smithi115 ceph-mon[36174]: osdmap e25: 4 total, 3 up, 4 in 2024-03-26T17:55:35.957 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:35.957 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:35.957 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:35 smithi115 ceph-mon[36174]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:35.957 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:35 smithi115 ceph-mon[36174]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' 2024-03-26T17:55:35.988 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:35 smithi049 ceph-mon[32331]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:55:35.988 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:35 smithi049 ceph-mon[32331]: osdmap e25: 4 total, 3 up, 4 in 2024-03-26T17:55:35.988 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:35.988 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:35.988 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:35 smithi049 ceph-mon[32331]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:55:35.988 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:35 smithi049 ceph-mon[32331]: from='osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844]' entity='osd.3' 2024-03-26T17:55:36.309 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2024-03-26T17:55:36.546 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:55:36.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:55:36.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:55:36.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:36.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] boot 2024-03-26T17:55:36.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T17:55:36.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:36.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi049 2024-03-26T17:55:36.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:36.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:36.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi049 to 8544M 2024-03-26T17:55:36.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:36.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:36.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.853 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:55:36.853 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:55:36.853 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:36.853 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] boot 2024-03-26T17:55:36.853 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi049 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi049 to 8544M 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:36.854 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:55:36.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:55:36.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:36.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: osd.3 [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] boot 2024-03-26T17:55:36.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T17:55:36.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:55:36.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi049 2024-03-26T17:55:36.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:36.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:36.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi049 to 8544M 2024-03-26T17:55:36.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:36.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:36.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:36.862 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:37.688 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:37 smithi115 ceph-mon[36174]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T17:55:37.688 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:37 smithi115 ceph-mon[36174]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:37.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:37 smithi171 ceph-mon[36033]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T17:55:37.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:37 smithi171 ceph-mon[36033]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:38.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:37 smithi049 ceph-mon[32331]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T17:55:38.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:37 smithi049 ceph-mon[32331]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:38.204 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.215+0000 7f337eec6700 1 -- 172.21.15.115:0/3218529027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33781006c0 msgr2=0x7f3378100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.215+0000 7f337eec6700 1 --2- 172.21.15.115:0/3218529027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33781006c0 0x7f3378100ae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f336c009990 tx=0x7f336c009ca0 comp rx=0 tx=0).stop 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.215+0000 7f337eec6700 1 -- 172.21.15.115:0/3218529027 shutdown_connections 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.215+0000 7f337eec6700 1 --2- 172.21.15.115:0/3218529027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3378104040 0x7f337810a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.215+0000 7f337eec6700 1 --2- 172.21.15.115:0/3218529027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33781006c0 0x7f3378100ae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.215+0000 7f337eec6700 1 --2- 172.21.15.115:0/3218529027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33781036a0 0x7f3378103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.215+0000 7f337eec6700 1 -- 172.21.15.115:0/3218529027 >> 172.21.15.115:0/3218529027 conn(0x7f33780fc040 msgr2=0x7f33780fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f337eec6700 1 -- 172.21.15.115:0/3218529027 shutdown_connections 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f337eec6700 1 -- 172.21.15.115:0/3218529027 wait complete. 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f337eec6700 1 Processor -- start 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f337eec6700 1 -- start start 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f337eec6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33781006c0 0x7f3378103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:38.205 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f337eec6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33781036a0 0x7f3378101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f337eec6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3378104040 0x7f3378101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f337eec6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f337810dfd0 con 0x7f3378104040 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f337eec6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f337810de50 con 0x7f33781036a0 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f337eec6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f337810e150 con 0x7f33781006c0 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f3377fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33781036a0 0x7f3378101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.216+0000 7f3377fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33781036a0 0x7f3378101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55458/0 (socket says 172.21.15.115:55458) 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f3377fff700 1 -- 172.21.15.115:0/2996940612 learned_addr learned my addr 172.21.15.115:0/2996940612 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f337cc62700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33781006c0 0x7f3378103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f337d463700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3378104040 0x7f3378101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f3377fff700 1 -- 172.21.15.115:0/2996940612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33781006c0 msgr2=0x7f3378103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f3377fff700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33781006c0 0x7f3378103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f3377fff700 1 -- 172.21.15.115:0/2996940612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3378104040 msgr2=0x7f3378101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:38.206 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f3377fff700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3378104040 0x7f3378101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:38.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f3377fff700 1 -- 172.21.15.115:0/2996940612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3378102440 con 0x7f33781036a0 2024-03-26T17:55:38.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f3377fff700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33781036a0 0x7f3378101650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f336400c920 tx=0x7f336400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:38.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f3375ffb700 1 -- 172.21.15.115:0/2996940612 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364007b00 con 0x7f33781036a0 2024-03-26T17:55:38.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f3375ffb700 1 -- 172.21.15.115:0/2996940612 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3364004d10 con 0x7f33781036a0 2024-03-26T17:55:38.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f337eec6700 1 -- 172.21.15.115:0/2996940612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33781a2780 con 0x7f33781036a0 2024-03-26T17:55:38.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f3375ffb700 1 -- 172.21.15.115:0/2996940612 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336400f450 con 0x7f33781036a0 2024-03-26T17:55:38.207 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.217+0000 7f337eec6700 1 -- 172.21.15.115:0/2996940612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33781a2c90 con 0x7f33781036a0 2024-03-26T17:55:38.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.218+0000 7f337eec6700 1 -- 172.21.15.115:0/2996940612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3378069070 con 0x7f33781036a0 2024-03-26T17:55:38.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.219+0000 7f3375ffb700 1 -- 172.21.15.115:0/2996940612 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f3364004750 con 0x7f33781036a0 2024-03-26T17:55:38.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.220+0000 7f3375ffb700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33680a2fb0 0x7f33680a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:38.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.220+0000 7f3375ffb700 1 -- 172.21.15.115:0/2996940612 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f33640ce240 con 0x7f33781036a0 2024-03-26T17:55:38.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.220+0000 7f337cc62700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33680a2fb0 0x7f33680a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:38.208 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.221+0000 7f337cc62700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33680a2fb0 0x7f33680a5470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f336c006010 tx=0x7f336c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:38.212 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.225+0000 7f3375ffb700 1 -- 172.21.15.115:0/2996940612 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3364014030 con 0x7f33781036a0 2024-03-26T17:55:38.419 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:38.432+0000 7f337eec6700 1 -- 172.21.15.115:0/2996940612 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f33780627f0 con 0x7f33680a2fb0 2024-03-26T17:55:38.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:38 smithi115 ceph-mon[36174]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T17:55:38.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:38 smithi115 ceph-mon[36174]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:55:38.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:38 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:55:38.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:38 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:55:38.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:38 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:38.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:38 smithi171 ceph-mon[36033]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T17:55:38.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:38 smithi171 ceph-mon[36033]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:55:38.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:38 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:55:38.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:38 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:55:38.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:38 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:39.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:38 smithi049 ceph-mon[32331]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T17:55:39.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:38 smithi049 ceph-mon[32331]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:55:39.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:38 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:55:39.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:38 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:55:39.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:38 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:39.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:39 smithi115 ceph-mon[36174]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T17:55:39.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:39 smithi115 ceph-mon[36174]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:39.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:39 smithi115 ceph-mon[36174]: osdmap e30: 4 total, 4 up, 4 in 2024-03-26T17:55:40.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:39 smithi049 ceph-mon[32331]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T17:55:40.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:39 smithi049 ceph-mon[32331]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:40.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:39 smithi049 ceph-mon[32331]: osdmap e30: 4 total, 4 up, 4 in 2024-03-26T17:55:40.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:39 smithi171 ceph-mon[36033]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T17:55:40.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:39 smithi171 ceph-mon[36033]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:40.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:39 smithi171 ceph-mon[36033]: osdmap e30: 4 total, 4 up, 4 in 2024-03-26T17:55:41.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:41 smithi115 ceph-mon[36174]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:41.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:41 smithi115 ceph-mon[36174]: from='client.? 172.21.15.115:0/1613369617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09354324-9abe-4d61-975b-7c82c3f57a2b"}]: dispatch 2024-03-26T17:55:41.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:41 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09354324-9abe-4d61-975b-7c82c3f57a2b"}]: dispatch 2024-03-26T17:55:41.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:41 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09354324-9abe-4d61-975b-7c82c3f57a2b"}]': finished 2024-03-26T17:55:41.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:41 smithi115 ceph-mon[36174]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T17:55:41.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:41 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:55:42.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:41 smithi049 ceph-mon[32331]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:42.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:41 smithi049 ceph-mon[32331]: from='client.? 172.21.15.115:0/1613369617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09354324-9abe-4d61-975b-7c82c3f57a2b"}]: dispatch 2024-03-26T17:55:42.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:41 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09354324-9abe-4d61-975b-7c82c3f57a2b"}]: dispatch 2024-03-26T17:55:42.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:41 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09354324-9abe-4d61-975b-7c82c3f57a2b"}]': finished 2024-03-26T17:55:42.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:41 smithi049 ceph-mon[32331]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T17:55:42.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:41 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:55:42.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:41 smithi171 ceph-mon[36033]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:42.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:41 smithi171 ceph-mon[36033]: from='client.? 172.21.15.115:0/1613369617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09354324-9abe-4d61-975b-7c82c3f57a2b"}]: dispatch 2024-03-26T17:55:42.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:41 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09354324-9abe-4d61-975b-7c82c3f57a2b"}]: dispatch 2024-03-26T17:55:42.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:41 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09354324-9abe-4d61-975b-7c82c3f57a2b"}]': finished 2024-03-26T17:55:42.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:41 smithi171 ceph-mon[36033]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T17:55:42.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:41 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:55:42.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:42 smithi115 ceph-mon[36174]: from='client.? 172.21.15.115:0/982885720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:55:43.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:42 smithi049 ceph-mon[32331]: from='client.? 172.21.15.115:0/982885720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:55:43.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:42 smithi171 ceph-mon[36033]: from='client.? 172.21.15.115:0/982885720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:55:43.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:43 smithi115 ceph-mon[36174]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-26T17:55:44.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:43 smithi049 ceph-mon[32331]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-26T17:55:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:43 smithi171 ceph-mon[36033]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-26T17:55:45.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:45 smithi115 ceph-mon[36174]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-26T17:55:46.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:45 smithi049 ceph-mon[32331]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-26T17:55:46.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:45 smithi171 ceph-mon[36033]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-26T17:55:47.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:47 smithi115 ceph-mon[36174]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T17:55:48.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:47 smithi049 ceph-mon[32331]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T17:55:48.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:47 smithi171 ceph-mon[36033]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T17:55:48.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:48 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T17:55:48.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:48 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:48.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:48 smithi115 ceph-mon[36174]: Deploying daemon osd.4 on smithi115 2024-03-26T17:55:49.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:48 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T17:55:49.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:48 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:49.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:48 smithi049 ceph-mon[32331]: Deploying daemon osd.4 on smithi115 2024-03-26T17:55:49.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:48 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T17:55:49.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:48 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:49.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:48 smithi171 ceph-mon[36033]: Deploying daemon osd.4 on smithi115 2024-03-26T17:55:49.833 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:49 smithi115 ceph-mon[36174]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-26T17:55:50.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:49 smithi049 ceph-mon[32331]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-26T17:55:50.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:49 smithi171 ceph-mon[36033]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-26T17:55:51.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:51 smithi115 ceph-mon[36174]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:55:52.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:51 smithi049 ceph-mon[32331]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:55:52.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:51 smithi171 ceph-mon[36033]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:55:53.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:53 smithi115 ceph-mon[36174]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-26T17:55:53.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:53.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:54.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:53 smithi049 ceph-mon[32331]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-26T17:55:54.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:54.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:54.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:53 smithi171 ceph-mon[36033]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-26T17:55:54.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:54.111 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:54.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:55.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:55.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:55.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:55 smithi115 ceph-mon[36174]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:56.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:55 smithi049 ceph-mon[32331]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:56.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:55 smithi171 ceph-mon[36033]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:57.434 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 4 on host 'smithi115' 2024-03-26T17:55:57.434 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.446+0000 7f3375ffb700 1 -- 172.21.15.115:0/2996940612 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f33780627f0 con 0x7f33680a2fb0 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 -- 172.21.15.115:0/2996940612 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33680a2fb0 msgr2=0x7f33680a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33680a2fb0 0x7f33680a5470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f336c006010 tx=0x7f336c00b540 comp rx=0 tx=0).stop 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 -- 172.21.15.115:0/2996940612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33781036a0 msgr2=0x7f3378101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33781036a0 0x7f3378101650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f336400c920 tx=0x7f336400cce0 comp rx=0 tx=0).stop 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 -- 172.21.15.115:0/2996940612 shutdown_connections 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33680a2fb0 0x7f33680a5470 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3378104040 0x7f3378101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33781006c0 0x7f3378103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 --2- 172.21.15.115:0/2996940612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33781036a0 0x7f3378101650 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 -- 172.21.15.115:0/2996940612 >> 172.21.15.115:0/2996940612 conn(0x7f33780fc040 msgr2=0x7f33781069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 -- 172.21.15.115:0/2996940612 shutdown_connections 2024-03-26T17:55:57.436 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:55:57.449+0000 7f337eec6700 1 -- 172.21.15.115:0/2996940612 wait complete. 2024-03-26T17:55:57.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:57.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:57.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:57.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:57.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:57.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:57.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:57.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:57.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:57.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:57.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:57.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:57.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:57.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:57.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:57.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:58.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:58.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:58.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:58.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:58.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:58.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:58.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:58.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:55:58.038 DEBUG:teuthology.orchestra.run.smithi115:osd.4> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.4.service 2024-03-26T17:55:58.041 INFO:tasks.cephadm:Deploying osd.5 on smithi171 with /dev/vg_nvme/lv_3... 2024-03-26T17:55:58.041 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T17:55:58.287 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:55:58.739 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:58 smithi171 ceph-mon[36033]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:58.739 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:58 smithi171 ceph-mon[36033]: from='osd.4 [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:55:58.740 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:58 smithi171 ceph-mon[36033]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:55:58.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:58 smithi115 ceph-mon[36174]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:58.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:58 smithi115 ceph-mon[36174]: from='osd.4 [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:55:58.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:58 smithi115 ceph-mon[36174]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:55:59.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:58 smithi049 ceph-mon[32331]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:55:59.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:58 smithi049 ceph-mon[32331]: from='osd.4 [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:55:59.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:58 smithi049 ceph-mon[32331]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:55:59.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:59 smithi115 ceph-mon[36174]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T17:55:59.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:59 smithi115 ceph-mon[36174]: from='osd.4 [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:55:59.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:59 smithi115 ceph-mon[36174]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T17:55:59.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:59 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:55:59.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:59 smithi115 ceph-mon[36174]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:55:59.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:55:59 smithi115 ceph-mon[36174]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:55:59.742 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 17:55:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4[47289]: 2024-03-26T17:55:59.546+0000 7fe1bf837700 -1 osd.4 0 waiting for initial osdmap 2024-03-26T17:55:59.742 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 17:55:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4[47289]: 2024-03-26T17:55:59.551+0000 7fe1bb62a700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:55:59.814 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:59 smithi171 ceph-mon[36033]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T17:55:59.814 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:59 smithi171 ceph-mon[36033]: from='osd.4 [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:55:59.814 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:59 smithi171 ceph-mon[36033]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T17:55:59.815 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:59 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:55:59.815 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:59 smithi171 ceph-mon[36033]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:55:59.815 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:55:59 smithi171 ceph-mon[36033]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:56:00.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:59 smithi049 ceph-mon[32331]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T17:56:00.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:59 smithi049 ceph-mon[32331]: from='osd.4 [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:56:00.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:59 smithi049 ceph-mon[32331]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T17:56:00.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:00.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:59 smithi049 ceph-mon[32331]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:56:00.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:55:59 smithi049 ceph-mon[32331]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:56:00.406 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T17:56:00.406 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T17:56:00.406 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-03-26T17:56:00.406 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-03-26T17:56:00.406 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011748 s, 893 MB/s 2024-03-26T17:56:00.406 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 2024-03-26T17:56:00.406 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-03-26T17:56:00.741 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:00 smithi171 ceph-mon[36033]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:56:00.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:00 smithi171 ceph-mon[36033]: osdmap e33: 5 total, 4 up, 5 in 2024-03-26T17:56:00.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:00 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:00.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:00 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:00.742 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:00 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:00.912 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:00 smithi115 ceph-mon[36174]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:56:00.912 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:00 smithi115 ceph-mon[36174]: osdmap e33: 5 total, 4 up, 5 in 2024-03-26T17:56:00.912 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:00 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:00.912 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:00 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:00.912 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:00 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:01.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:00 smithi049 ceph-mon[32331]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:56:01.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:00 smithi049 ceph-mon[32331]: osdmap e33: 5 total, 4 up, 5 in 2024-03-26T17:56:01.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:00 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:01.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:00 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:01.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:00 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:01.086 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi171:vg_nvme/lv_3 2024-03-26T17:56:01.333 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:56:01.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:56:01.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:56:01.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: osd.4 [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] boot 2024-03-26T17:56:01.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T17:56:01.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:01.646 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:01.647 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:01.647 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:01.647 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:01.647 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:01.647 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:01.647 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:01.647 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:01.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:56:01.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: osd.4 [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] boot 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:01.992 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:02.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:56:02.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:56:02.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: osd.4 [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] boot 2024-03-26T17:56:02.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T17:56:02.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:56:02.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:02.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:02.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:02.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:02.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:02.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:02.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:02.007 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:02.727 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:02 smithi171 ceph-mon[36033]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:56:02.727 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:02 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi115 2024-03-26T17:56:02.727 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:02 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi115 to 8540M 2024-03-26T17:56:02.728 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:02 smithi171 ceph-mon[36033]: osdmap e35: 5 total, 5 up, 5 in 2024-03-26T17:56:02.940 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 -- 172.21.15.171:0/1224447478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d0101310 msgr2=0x7fc3d0101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:02.940 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 --2- 172.21.15.171:0/1224447478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d0101310 0x7fc3d0101730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40098f0 tx=0x7fc3c4009c00 comp rx=0 tx=0).stop 2024-03-26T17:56:02.940 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 -- 172.21.15.171:0/1224447478 shutdown_connections 2024-03-26T17:56:02.940 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 --2- 172.21.15.171:0/1224447478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d0102eb0 0x7fc3d0109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:02.940 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 --2- 172.21.15.171:0/1224447478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d0101310 0x7fc3d0101730 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 --2- 172.21.15.171:0/1224447478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3d0102510 0x7fc3d0102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 -- 172.21.15.171:0/1224447478 >> 172.21.15.171:0/1224447478 conn(0x7fc3d00fc8d0 msgr2=0x7fc3d00fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 -- 172.21.15.171:0/1224447478 shutdown_connections 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 -- 172.21.15.171:0/1224447478 wait complete. 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 Processor -- start 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 -- start start 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3d0101310 0x7fc3d0197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d0102510 0x7fc3d0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d0102eb0 0x7fc3d019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d010cf00 con 0x7fc3d0102510 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d010cd80 con 0x7fc3d0101310 2024-03-26T17:56:02.941 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.952+0000 7fc3d7018700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d010d080 con 0x7fc3d0102eb0 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d55b5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d0102eb0 0x7fc3d019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d55b5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d0102eb0 0x7fc3d019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:59494/0 (socket says 172.21.15.171:59494) 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d55b5700 1 -- 172.21.15.171:0/1843120572 learned_addr learned my addr 172.21.15.171:0/1843120572 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3cffff700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d0102510 0x7fc3d0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d4db4700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3d0101310 0x7fc3d0197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d55b5700 1 -- 172.21.15.171:0/1843120572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3d0101310 msgr2=0x7fc3d0197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d55b5700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3d0101310 0x7fc3d0197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d55b5700 1 -- 172.21.15.171:0/1843120572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d0102510 msgr2=0x7fc3d0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d55b5700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d0102510 0x7fc3d0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d55b5700 1 -- 172.21.15.171:0/1843120572 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d019f1f0 con 0x7fc3d0102eb0 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d55b5700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d0102eb0 0x7fc3d019eb50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800c140 tx=0x7fc3c800c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3cdffb700 1 -- 172.21.15.171:0/1843120572 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c800cdc0 con 0x7fc3d0102eb0 2024-03-26T17:56:02.942 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3cdffb700 1 -- 172.21.15.171:0/1843120572 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c8004bb0 con 0x7fc3d0102eb0 2024-03-26T17:56:02.943 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d7018700 1 -- 172.21.15.171:0/1843120572 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d019f480 con 0x7fc3d0102eb0 2024-03-26T17:56:02.943 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3cdffb700 1 -- 172.21.15.171:0/1843120572 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8007e30 con 0x7fc3d0102eb0 2024-03-26T17:56:02.943 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.953+0000 7fc3d7018700 1 -- 172.21.15.171:0/1843120572 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d019f9c0 con 0x7fc3d0102eb0 2024-03-26T17:56:02.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.954+0000 7fc3d7018700 1 -- 172.21.15.171:0/1843120572 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d0069070 con 0x7fc3d0102eb0 2024-03-26T17:56:02.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.955+0000 7fc3cdffb700 1 -- 172.21.15.171:0/1843120572 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fc3c801a020 con 0x7fc3d0102eb0 2024-03-26T17:56:02.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.956+0000 7fc3cdffb700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc3c00a2fb0 0x7fc3c00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:02.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.956+0000 7fc3cdffb700 1 -- 172.21.15.171:0/1843120572 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fc3c80cdf10 con 0x7fc3d0102eb0 2024-03-26T17:56:02.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.956+0000 7fc3d4db4700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc3c00a2fb0 0x7fc3c00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:02.944 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.957+0000 7fc3d4db4700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc3c00a2fb0 0x7fc3c00a5470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009810 tx=0x7fc3c4009f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:02.947 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:02.961+0000 7fc3cdffb700 1 -- 172.21.15.171:0/1843120572 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc3c8010040 con 0x7fc3d0102eb0 2024-03-26T17:56:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:02 smithi115 ceph-mon[36174]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:56:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:02 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi115 2024-03-26T17:56:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:02 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi115 to 8540M 2024-03-26T17:56:02.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:02 smithi115 ceph-mon[36174]: osdmap e35: 5 total, 5 up, 5 in 2024-03-26T17:56:03.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:02 smithi049 ceph-mon[32331]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:56:03.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:02 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi115 2024-03-26T17:56:03.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:02 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi115 to 8540M 2024-03-26T17:56:03.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:02 smithi049 ceph-mon[32331]: osdmap e35: 5 total, 5 up, 5 in 2024-03-26T17:56:03.151 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:03.165+0000 7fc3d7018700 1 -- 172.21.15.171:0/1843120572 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc3d00627f0 con 0x7fc3c00a2fb0 2024-03-26T17:56:03.819 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:03 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:56:03.819 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:03 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:56:03.819 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:03 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:03.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:03 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:56:03.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:03 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:56:03.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:03 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:04.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:03 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:56:04.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:03 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:56:04.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:03 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:04.837 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:04 smithi171 ceph-mon[36033]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:04.838 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:04 smithi171 ceph-mon[36033]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:56:04.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:04 smithi115 ceph-mon[36174]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:04.892 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:04 smithi115 ceph-mon[36174]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:56:04.907 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:04 smithi049 ceph-mon[32331]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:04.907 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:04 smithi049 ceph-mon[32331]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:56:05.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:05 smithi171 ceph-mon[36033]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:05.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:05 smithi115 ceph-mon[36174]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:06.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:05 smithi049 ceph-mon[32331]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:06.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:06 smithi171 ceph-mon[36033]: from='client.? 172.21.15.171:0/3605491906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89af3500-86a4-4dcc-a44d-35ba9af96293"}]: dispatch 2024-03-26T17:56:06.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:06 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89af3500-86a4-4dcc-a44d-35ba9af96293"}]: dispatch 2024-03-26T17:56:06.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:06 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89af3500-86a4-4dcc-a44d-35ba9af96293"}]': finished 2024-03-26T17:56:06.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:06 smithi171 ceph-mon[36033]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T17:56:06.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:06 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:06.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:06 smithi171 ceph-mon[36033]: from='client.? 172.21.15.171:0/3880391472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:56:06.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:06 smithi115 ceph-mon[36174]: from='client.? 172.21.15.171:0/3605491906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89af3500-86a4-4dcc-a44d-35ba9af96293"}]: dispatch 2024-03-26T17:56:06.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:06 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89af3500-86a4-4dcc-a44d-35ba9af96293"}]: dispatch 2024-03-26T17:56:06.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:06 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89af3500-86a4-4dcc-a44d-35ba9af96293"}]': finished 2024-03-26T17:56:06.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:06 smithi115 ceph-mon[36174]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T17:56:06.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:06 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:06.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:06 smithi115 ceph-mon[36174]: from='client.? 172.21.15.171:0/3880391472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:56:07.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:06 smithi049 ceph-mon[32331]: from='client.? 172.21.15.171:0/3605491906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89af3500-86a4-4dcc-a44d-35ba9af96293"}]: dispatch 2024-03-26T17:56:07.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:06 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89af3500-86a4-4dcc-a44d-35ba9af96293"}]: dispatch 2024-03-26T17:56:07.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:06 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89af3500-86a4-4dcc-a44d-35ba9af96293"}]': finished 2024-03-26T17:56:07.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:06 smithi049 ceph-mon[32331]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T17:56:07.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:06 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:07.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:06 smithi049 ceph-mon[32331]: from='client.? 172.21.15.171:0/3880391472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:56:07.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:07 smithi171 ceph-mon[36033]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:07.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:07 smithi115 ceph-mon[36174]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:08.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:07 smithi049 ceph-mon[32331]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:09.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:09 smithi115 ceph-mon[36174]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:10.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:09 smithi049 ceph-mon[32331]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:10.017 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:09 smithi171 ceph-mon[36033]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:11.978 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:11 smithi171 ceph-mon[36033]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:11.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:11 smithi115 ceph-mon[36174]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:12.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:11 smithi049 ceph-mon[32331]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:13.838 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:13 smithi171 ceph-mon[36033]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:13.838 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T17:56:13.838 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:13.838 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:13 smithi171 ceph-mon[36033]: Deploying daemon osd.5 on smithi171 2024-03-26T17:56:13.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:13 smithi115 ceph-mon[36174]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:13.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T17:56:13.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:13.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:13 smithi115 ceph-mon[36174]: Deploying daemon osd.5 on smithi171 2024-03-26T17:56:14.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:13 smithi049 ceph-mon[32331]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:14.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T17:56:14.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:14.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:13 smithi049 ceph-mon[32331]: Deploying daemon osd.5 on smithi171 2024-03-26T17:56:15.986 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:15 smithi171 ceph-mon[36033]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:15.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:15 smithi115 ceph-mon[36174]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:16.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:15 smithi049 ceph-mon[32331]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:17.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:17 smithi171 ceph-mon[36033]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:17.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:17 smithi115 ceph-mon[36174]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:17 smithi049 ceph-mon[32331]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:19.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:19.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:19.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:19.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:19.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:19.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:19.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:19.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:19.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:20.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:20 smithi171 ceph-mon[36033]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:20.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:20 smithi115 ceph-mon[36174]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:20.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:20 smithi049 ceph-mon[32331]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:22.015 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.028+0000 7fc3cdffb700 1 -- 172.21.15.171:0/1843120572 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc3d00627f0 con 0x7fc3c00a2fb0 2024-03-26T17:56:22.016 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 5 on host 'smithi171' 2024-03-26T17:56:22.018 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 -- 172.21.15.171:0/1843120572 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc3c00a2fb0 msgr2=0x7fc3c00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:22.018 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc3c00a2fb0 0x7fc3c00a5470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009810 tx=0x7fc3c4009f70 comp rx=0 tx=0).stop 2024-03-26T17:56:22.018 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 -- 172.21.15.171:0/1843120572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d0102eb0 msgr2=0x7fc3d019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:22.018 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d0102eb0 0x7fc3d019eb50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800c140 tx=0x7fc3c800c500 comp rx=0 tx=0).stop 2024-03-26T17:56:22.018 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 -- 172.21.15.171:0/1843120572 shutdown_connections 2024-03-26T17:56:22.018 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc3c00a2fb0 0x7fc3c00a5470 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:22.018 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d0102eb0 0x7fc3d019eb50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:22.019 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3d0101310 0x7fc3d0197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:22.019 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 --2- 172.21.15.171:0/1843120572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3d0102510 0x7fc3d0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:22.019 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 -- 172.21.15.171:0/1843120572 >> 172.21.15.171:0/1843120572 conn(0x7fc3d00fc8d0 msgr2=0x7fc3d01058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:22.019 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 -- 172.21.15.171:0/1843120572 shutdown_connections 2024-03-26T17:56:22.019 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:56:22.031+0000 7fc3d7018700 1 -- 172.21.15.171:0/1843120572 wait complete. 2024-03-26T17:56:22.230 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:22 smithi171 ceph-mon[36033]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:22.230 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.230 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.230 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:22.230 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:22.230 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.230 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.231 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.231 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:22.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:22 smithi115 ceph-mon[36174]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:22.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:22.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:22.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:22.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:22 smithi049 ceph-mon[32331]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:22.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:22.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:22.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:22.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:22.591 DEBUG:teuthology.orchestra.run.smithi171:osd.5> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.5.service 2024-03-26T17:56:22.593 INFO:tasks.cephadm:Deploying osd.6 on smithi049 with /dev/vg_nvme/lv_2... 2024-03-26T17:56:22.594 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T17:56:22.852 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:56:23.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:23 smithi049 ceph-mon[32331]: from='osd.5 [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:56:23.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:23 smithi049 ceph-mon[32331]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:56:23.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:23 smithi171 ceph-mon[36033]: from='osd.5 [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:56:23.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:23 smithi171 ceph-mon[36033]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:56:23.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:23 smithi115 ceph-mon[36174]: from='osd.5 [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:56:23.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:23 smithi115 ceph-mon[36174]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:56:24.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:24 smithi171 ceph-mon[36033]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:24.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:24 smithi171 ceph-mon[36033]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T17:56:24.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:24 smithi171 ceph-mon[36033]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T17:56:24.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:24.360 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:24 smithi171 ceph-mon[36033]: from='osd.5 [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:56:24.361 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:24 smithi171 ceph-mon[36033]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:56:24.361 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 17:56:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T17:56:24.158+0000 7f856eb8e700 -1 osd.5 0 waiting for initial osdmap 2024-03-26T17:56:24.361 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 17:56:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T17:56:24.165+0000 7f856a981700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:56:24.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:24 smithi049 ceph-mon[32331]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:24.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:24 smithi049 ceph-mon[32331]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T17:56:24.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:24 smithi049 ceph-mon[32331]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T17:56:24.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:24.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:24 smithi049 ceph-mon[32331]: from='osd.5 [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:56:24.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:24 smithi049 ceph-mon[32331]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:56:24.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:24 smithi115 ceph-mon[36174]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:24.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:24 smithi115 ceph-mon[36174]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T17:56:24.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:24 smithi115 ceph-mon[36174]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T17:56:24.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:24.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:24 smithi115 ceph-mon[36174]: from='osd.5 [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:56:24.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:24 smithi115 ceph-mon[36174]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:56:25.036 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T17:56:25.036 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T17:56:25.036 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-26T17:56:25.036 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-26T17:56:25.036 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00866214 s, 1.2 GB/s 2024-03-26T17:56:25.036 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-26T17:56:25.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:25 smithi049 ceph-mon[32331]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:56:25.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:25 smithi049 ceph-mon[32331]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T17:56:25.345 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:25 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:25.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:25 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:25.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:25 smithi049 ceph-mon[32331]: osd.5 [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] boot 2024-03-26T17:56:25.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:25 smithi049 ceph-mon[32331]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T17:56:25.346 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:25 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:25.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:25 smithi115 ceph-mon[36174]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:56:25.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:25 smithi115 ceph-mon[36174]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T17:56:25.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:25 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:25.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:25 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:25.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:25 smithi115 ceph-mon[36174]: osd.5 [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] boot 2024-03-26T17:56:25.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:25 smithi115 ceph-mon[36174]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T17:56:25.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:25 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:25.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:25 smithi171 ceph-mon[36033]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:56:25.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:25 smithi171 ceph-mon[36033]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T17:56:25.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:25 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:25.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:25 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:25.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:25 smithi171 ceph-mon[36033]: osd.5 [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] boot 2024-03-26T17:56:25.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:25 smithi171 ceph-mon[36033]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T17:56:25.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:25 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:56:25.695 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_2 2024-03-26T17:56:25.948 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:56:26.343 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi171 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi171 to 8540M 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.344 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:26 smithi049 ceph-mon[32331]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi171 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi171 to 8540M 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:26.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:26.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.492 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:26 smithi115 ceph-mon[36174]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T17:56:26.559 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:56:26.559 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:56:26.559 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:56:26.559 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi171 2024-03-26T17:56:26.559 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.559 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.559 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:26.559 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:26.559 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi171 to 8540M 2024-03-26T17:56:26.560 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.560 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:26.560 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:26.560 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:26.560 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:26 smithi171 ceph-mon[36033]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T17:56:27.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.705+0000 7f11d80ab700 1 -- 172.21.15.49:0/2299376281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d01012e0 msgr2=0x7f11d0101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:27.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.705+0000 7f11d80ab700 1 --2- 172.21.15.49:0/2299376281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d01012e0 0x7f11d0101700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0099c0 tx=0x7f11cc009cd0 comp rx=0 tx=0).stop 2024-03-26T17:56:27.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.705+0000 7f11d80ab700 1 -- 172.21.15.49:0/2299376281 shutdown_connections 2024-03-26T17:56:27.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.705+0000 7f11d80ab700 1 --2- 172.21.15.49:0/2299376281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d0102e80 0x7f11d0109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:27.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.705+0000 7f11d80ab700 1 --2- 172.21.15.49:0/2299376281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d01012e0 0x7f11d0101700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.705+0000 7f11d80ab700 1 --2- 172.21.15.49:0/2299376281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11d01024e0 0x7f11d0102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.705+0000 7f11d80ab700 1 -- 172.21.15.49:0/2299376281 >> 172.21.15.49:0/2299376281 conn(0x7f11d00fc880 msgr2=0x7f11d00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.705+0000 7f11d80ab700 1 -- 172.21.15.49:0/2299376281 shutdown_connections 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.705+0000 7f11d80ab700 1 -- 172.21.15.49:0/2299376281 wait complete. 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d80ab700 1 Processor -- start 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d80ab700 1 -- start start 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d80ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d01012e0 0x7f11d006abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d80ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d01024e0 0x7f11d019d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d80ab700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11d0102e80 0x7f11d019f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d80ab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11d010cec0 con 0x7f11d01024e0 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d80ab700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f11d010cd40 con 0x7f11d01012e0 2024-03-26T17:56:27.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d80ab700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f11d010d040 con 0x7f11d0102e80 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d5646700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d01024e0 0x7f11d019d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d5646700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d01024e0 0x7f11d019d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53372/0 (socket says 172.21.15.49:53372) 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d5e47700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d01012e0 0x7f11d006abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.706+0000 7f11d5646700 1 -- 172.21.15.49:0/2975996366 learned_addr learned my addr 172.21.15.49:0/2975996366 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11d6648700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11d0102e80 0x7f11d019f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11d5646700 1 -- 172.21.15.49:0/2975996366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11d0102e80 msgr2=0x7f11d019f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11d5646700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11d0102e80 0x7f11d019f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11d5646700 1 -- 172.21.15.49:0/2975996366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d01012e0 msgr2=0x7f11d006abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11d5646700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d01012e0 0x7f11d006abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11d5646700 1 -- 172.21.15.49:0/2975996366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d01a0270 con 0x7f11d01024e0 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11d5646700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d01024e0 0x7f11d019d360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11c000c970 tx=0x7f11c000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11c6ffd700 1 -- 172.21.15.49:0/2975996366 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0007aa0 con 0x7f11d01024e0 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11d80ab700 1 -- 172.21.15.49:0/2975996366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d01a0560 con 0x7f11d01024e0 2024-03-26T17:56:27.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11c6ffd700 1 -- 172.21.15.49:0/2975996366 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c0004d10 con 0x7f11d01024e0 2024-03-26T17:56:27.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11c6ffd700 1 -- 172.21.15.49:0/2975996366 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c000f450 con 0x7f11d01024e0 2024-03-26T17:56:27.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.707+0000 7f11d80ab700 1 -- 172.21.15.49:0/2975996366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d01a0aa0 con 0x7f11d01024e0 2024-03-26T17:56:27.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.708+0000 7f11d80ab700 1 -- 172.21.15.49:0/2975996366 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d0069070 con 0x7f11d01024e0 2024-03-26T17:56:27.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.709+0000 7f11c6ffd700 1 -- 172.21.15.49:0/2975996366 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f11c0004750 con 0x7f11d01024e0 2024-03-26T17:56:27.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.710+0000 7f11c6ffd700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f11bc0a3030 0x7f11bc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:27.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.710+0000 7f11d5e47700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f11bc0a3030 0x7f11bc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:27.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.710+0000 7f11c6ffd700 1 -- 172.21.15.49:0/2975996366 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f11c00ce290 con 0x7f11d01024e0 2024-03-26T17:56:27.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.711+0000 7f11d5e47700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f11bc0a3030 0x7f11bc0a54f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f11cc000c00 tx=0x7f11cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:27.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.715+0000 7f11c6ffd700 1 -- 172.21.15.49:0/2975996366 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f11c009c010 con 0x7f11d01024e0 2024-03-26T17:56:27.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:27.930+0000 7f11d80ab700 1 -- 172.21.15.49:0/2975996366 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f11d002e050 con 0x7f11bc0a3030 2024-03-26T17:56:28.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:28 smithi115 ceph-mon[36174]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:28.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:28 smithi115 ceph-mon[36174]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T17:56:28.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:28 smithi115 ceph-mon[36174]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T17:56:28.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:28 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:56:28.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:28 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:56:28.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:28 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:28.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:28 smithi049 ceph-mon[32331]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:28.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:28 smithi049 ceph-mon[32331]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T17:56:28.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:28 smithi049 ceph-mon[32331]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T17:56:28.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:28 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:56:28.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:28 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:56:28.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:28 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:28 smithi171 ceph-mon[36033]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:28 smithi171 ceph-mon[36033]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T17:56:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:28 smithi171 ceph-mon[36033]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T17:56:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:56:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:56:28.610 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:29.486 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:29 smithi049 ceph-mon[32331]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:56:29.486 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:29 smithi049 ceph-mon[32331]: osdmap e43: 6 total, 6 up, 6 in 2024-03-26T17:56:29.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:29 smithi115 ceph-mon[36174]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:56:29.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:29 smithi115 ceph-mon[36174]: osdmap e43: 6 total, 6 up, 6 in 2024-03-26T17:56:29.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:29 smithi171 ceph-mon[36033]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:56:29.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:29 smithi171 ceph-mon[36033]: osdmap e43: 6 total, 6 up, 6 in 2024-03-26T17:56:30.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:30 smithi115 ceph-mon[36174]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:30.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:30 smithi049 ceph-mon[32331]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:30.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:30 smithi171 ceph-mon[36033]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:31.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:31 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/9726425' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93c76579-30ec-4c2d-a96a-7f939a9cfd3c"}]: dispatch 2024-03-26T17:56:31.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:31 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/9726425' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93c76579-30ec-4c2d-a96a-7f939a9cfd3c"}]': finished 2024-03-26T17:56:31.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:31 smithi115 ceph-mon[36174]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T17:56:31.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:31 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:31.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:31 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4002458141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:56:31.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:31 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/9726425' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93c76579-30ec-4c2d-a96a-7f939a9cfd3c"}]: dispatch 2024-03-26T17:56:31.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:31 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/9726425' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93c76579-30ec-4c2d-a96a-7f939a9cfd3c"}]': finished 2024-03-26T17:56:31.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:31 smithi049 ceph-mon[32331]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T17:56:31.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:31 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:31.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:31 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4002458141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:56:31.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:31 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/9726425' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93c76579-30ec-4c2d-a96a-7f939a9cfd3c"}]: dispatch 2024-03-26T17:56:31.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:31 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/9726425' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93c76579-30ec-4c2d-a96a-7f939a9cfd3c"}]': finished 2024-03-26T17:56:31.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:31 smithi171 ceph-mon[36033]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T17:56:31.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:31 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:31.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:31 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4002458141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:56:32.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:32 smithi115 ceph-mon[36174]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:32.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:32 smithi049 ceph-mon[32331]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:32.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:32 smithi171 ceph-mon[36033]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:34.703 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:34 smithi049 ceph-mon[32331]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-03-26T17:56:34.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:34 smithi115 ceph-mon[36174]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-03-26T17:56:34.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:34 smithi171 ceph-mon[36033]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-03-26T17:56:36.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:36 smithi115 ceph-mon[36174]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T17:56:36.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:36 smithi049 ceph-mon[32331]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T17:56:36.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:36 smithi171 ceph-mon[36033]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T17:56:38.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:38 smithi049 ceph-mon[32331]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T17:56:38.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:38 smithi115 ceph-mon[36174]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T17:56:38.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:38 smithi171 ceph-mon[36033]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T17:56:39.490 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:39 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T17:56:39.490 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:39 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:39.490 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:39 smithi049 ceph-mon[32331]: Deploying daemon osd.6 on smithi049 2024-03-26T17:56:39.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:39 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T17:56:39.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:39 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:39.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:39 smithi115 ceph-mon[36174]: Deploying daemon osd.6 on smithi049 2024-03-26T17:56:39.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:39 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T17:56:39.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:39 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:39.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:39 smithi171 ceph-mon[36033]: Deploying daemon osd.6 on smithi049 2024-03-26T17:56:40.471 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:40 smithi049 ceph-mon[32331]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:56:40.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:40 smithi115 ceph-mon[36174]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:56:40.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:40 smithi171 ceph-mon[36033]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:56:42.506 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:42 smithi049 ceph-mon[32331]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:56:42.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:42 smithi115 ceph-mon[36174]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:56:42.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:42 smithi171 ceph-mon[36033]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:56:44.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:44 smithi115 ceph-mon[36174]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:56:44.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:44 smithi049 ceph-mon[32331]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:56:44.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:44 smithi171 ceph-mon[36033]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:56:45.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:45 smithi115 ceph-mon[36174]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:45.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:45.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:45.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:46.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:45 smithi049 ceph-mon[32331]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:46.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:46.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:46.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:46.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:45 smithi171 ceph-mon[36033]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:46.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:46.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:46.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:47.979 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:47 smithi049 ceph-mon[32331]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:47.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:47 smithi115 ceph-mon[36174]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:48.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:47 smithi171 ceph-mon[36033]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:48.494 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 6 on host 'smithi049' 2024-03-26T17:56:48.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.508+0000 7f11c6ffd700 1 -- 172.21.15.49:0/2975996366 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f11d002e050 con 0x7f11bc0a3030 2024-03-26T17:56:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 -- 172.21.15.49:0/2975996366 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f11bc0a3030 msgr2=0x7f11bc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f11bc0a3030 0x7f11bc0a54f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f11cc000c00 tx=0x7f11cc005fb0 comp rx=0 tx=0).stop 2024-03-26T17:56:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 -- 172.21.15.49:0/2975996366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d01024e0 msgr2=0x7f11d019d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d01024e0 0x7f11d019d360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11c000c970 tx=0x7f11c000cc80 comp rx=0 tx=0).stop 2024-03-26T17:56:48.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 -- 172.21.15.49:0/2975996366 shutdown_connections 2024-03-26T17:56:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f11bc0a3030 0x7f11bc0a54f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11d0102e80 0x7f11d019f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11d01012e0 0x7f11d006abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 --2- 172.21.15.49:0/2975996366 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11d01024e0 0x7f11d019d360 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 -- 172.21.15.49:0/2975996366 >> 172.21.15.49:0/2975996366 conn(0x7f11d00fc880 msgr2=0x7f11d01057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 -- 172.21.15.49:0/2975996366 shutdown_connections 2024-03-26T17:56:48.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:56:48.511+0000 7f11d80ab700 1 -- 172.21.15.49:0/2975996366 wait complete. 2024-03-26T17:56:49.156 DEBUG:teuthology.orchestra.run.smithi049:osd.6> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.6.service 2024-03-26T17:56:49.159 INFO:tasks.cephadm:Deploying osd.7 on smithi115 with /dev/vg_nvme/lv_2... 2024-03-26T17:56:49.159 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T17:56:49.405 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:56:49.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:49 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:49 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:49 smithi115 ceph-mon[36174]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T17:56:49.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:49 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:49 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:49 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:49 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:49 smithi049 ceph-mon[32331]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T17:56:49.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:49 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:49 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:49 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:49 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:49 smithi171 ceph-mon[36033]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T17:56:49.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:49 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:49.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:49 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:50.721 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:50 smithi115 ceph-mon[36174]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:50.721 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:50 smithi115 ceph-mon[36174]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T17:56:50.721 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:50 smithi115 ceph-mon[36174]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T17:56:50.721 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:50 smithi115 ceph-mon[36174]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:56:50.721 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:50 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:50.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:50 smithi049 ceph-mon[32331]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:50.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:50 smithi049 ceph-mon[32331]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T17:56:50.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:50 smithi049 ceph-mon[32331]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T17:56:50.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:50 smithi049 ceph-mon[32331]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:56:50.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:50 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:50.756 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 17:56:50 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T17:56:50.477+0000 7fdeb4fa6700 -1 osd.6 0 waiting for initial osdmap 2024-03-26T17:56:50.756 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 17:56:50 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T17:56:50.486+0000 7fdeb0d99700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:56:50.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:50 smithi171 ceph-mon[36033]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:50.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:50 smithi171 ceph-mon[36033]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T17:56:50.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:50 smithi171 ceph-mon[36033]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T17:56:50.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:50 smithi171 ceph-mon[36033]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:56:50.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:50 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:51.405 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T17:56:51.405 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T17:56:51.405 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-03-26T17:56:51.405 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-03-26T17:56:51.405 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148582 s, 706 MB/s 2024-03-26T17:56:51.405 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-03-26T17:56:51.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:51 smithi115 ceph-mon[36174]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:56:51.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:51 smithi115 ceph-mon[36174]: osdmap e46: 7 total, 6 up, 7 in 2024-03-26T17:56:51.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:51.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:51 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:51.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:51 smithi049 ceph-mon[32331]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:56:51.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:51 smithi049 ceph-mon[32331]: osdmap e46: 7 total, 6 up, 7 in 2024-03-26T17:56:51.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:51.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:51 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:51.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:51 smithi171 ceph-mon[36033]: from='osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:56:51.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:51 smithi171 ceph-mon[36033]: osdmap e46: 7 total, 6 up, 7 in 2024-03-26T17:56:51.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:51.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:51 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:52.113 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2024-03-26T17:56:52.352 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:56:52.717 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:52 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:56:52.718 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:52 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:56:52.718 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:52 smithi115 ceph-mon[36174]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:52.718 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:52 smithi115 ceph-mon[36174]: osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] boot 2024-03-26T17:56:52.718 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:52 smithi115 ceph-mon[36174]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T17:56:52.718 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:52 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:52.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:52 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:56:52.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:52 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:56:52.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:52 smithi049 ceph-mon[32331]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:52.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:52 smithi049 ceph-mon[32331]: osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] boot 2024-03-26T17:56:52.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:52 smithi049 ceph-mon[32331]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T17:56:52.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:52 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:52.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:52 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:56:52.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:52 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:56:52.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:52 smithi171 ceph-mon[36033]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:56:52.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:52 smithi171 ceph-mon[36033]: osd.6 [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] boot 2024-03-26T17:56:52.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:52 smithi171 ceph-mon[36033]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T17:56:52.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:52 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:56:53.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: osdmap e48: 7 total, 7 up, 7 in 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:53.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:53 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: osdmap e48: 7 total, 7 up, 7 in 2024-03-26T17:56:53.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:53.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:53 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: osdmap e48: 7 total, 7 up, 7 in 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:53.761 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:53.762 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:53.762 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:53 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.030+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3066670495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb404101f30 msgr2=0x7fb4041097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.030+0000 7fb40bfe4700 1 --2- 172.21.15.115:0/3066670495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb404101f30 0x7fb4041097d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00b0a0 tx=0x7fb3fc00b3b0 comp rx=0 tx=0).stop 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.030+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3066670495 shutdown_connections 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.030+0000 7fb40bfe4700 1 --2- 172.21.15.115:0/3066670495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb404101f30 0x7fb4041097d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 --2- 172.21.15.115:0/3066670495 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb404102910 0x7fb404102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 --2- 172.21.15.115:0/3066670495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb404101570 0x7fb4041019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3066670495 >> 172.21.15.115:0/3066670495 conn(0x7fb4040fc000 msgr2=0x7fb4040fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3066670495 shutdown_connections 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3066670495 wait complete. 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 Processor -- start 2024-03-26T17:56:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 -- start start 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb404101570 0x7fb4041985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb404101f30 0x7fb404198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb404102910 0x7fb40419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410e030 con 0x7fb404102910 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410deb0 con 0x7fb404101570 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb40bfe4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410e1b0 con 0x7fb404101f30 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb409d80700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb404101570 0x7fb4041985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb409d80700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb404101570 0x7fb4041985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59774/0 (socket says 172.21.15.115:59774) 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.031+0000 7fb409d80700 1 -- 172.21.15.115:0/3940500582 learned_addr learned my addr 172.21.15.115:0/3940500582 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb40957f700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb404101f30 0x7fb404198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb40a581700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb404102910 0x7fb40419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb409d80700 1 -- 172.21.15.115:0/3940500582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb404101f30 msgr2=0x7fb404198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:54.020 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb409d80700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb404101f30 0x7fb404198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb409d80700 1 -- 172.21.15.115:0/3940500582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb404102910 msgr2=0x7fb40419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb409d80700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb404102910 0x7fb40419fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb409d80700 1 -- 172.21.15.115:0/3940500582 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4041a0210 con 0x7fb404101570 2024-03-26T17:56:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb409d80700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb404101570 0x7fb4041985b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400d980 tx=0x7fb3f400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb3faffd700 1 -- 172.21.15.115:0/3940500582 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4017030 con 0x7fb404101570 2024-03-26T17:56:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3940500582 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4041a0500 con 0x7fb404101570 2024-03-26T17:56:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb3faffd700 1 -- 172.21.15.115:0/3940500582 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f4004500 con 0x7fb404101570 2024-03-26T17:56:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb3faffd700 1 -- 172.21.15.115:0/3940500582 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4003b20 con 0x7fb404101570 2024-03-26T17:56:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.032+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3940500582 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041a0a90 con 0x7fb404101570 2024-03-26T17:56:54.021 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.033+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3940500582 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb404069070 con 0x7fb404101570 2024-03-26T17:56:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.034+0000 7fb3faffd700 1 -- 172.21.15.115:0/3940500582 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fb3f401a070 con 0x7fb404101570 2024-03-26T17:56:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.035+0000 7fb3faffd700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3f00a2fb0 0x7fb3f00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.035+0000 7fb3faffd700 1 -- 172.21.15.115:0/3940500582 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fb3f40ce470 con 0x7fb404101570 2024-03-26T17:56:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.035+0000 7fb40957f700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3f00a2fb0 0x7fb3f00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.035+0000 7fb40957f700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3f00a2fb0 0x7fb3f00a5470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb400005950 tx=0x7fb4000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:54.024 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.038+0000 7fb3faffd700 1 -- 172.21.15.115:0/3940500582 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb3f4009940 con 0x7fb404101570 2024-03-26T17:56:54.229 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:56:54.243+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3940500582 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb4040627f0 con 0x7fb3f00a2fb0 2024-03-26T17:56:54.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:54 smithi115 ceph-mon[36174]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:56:54.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:54 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi049 2024-03-26T17:56:54.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:54 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi049 to 5696M 2024-03-26T17:56:54.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:56:54.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:56:54.491 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:54.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:54 smithi049 ceph-mon[32331]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:56:54.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:54 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi049 2024-03-26T17:56:54.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:54 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi049 to 5696M 2024-03-26T17:56:54.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:56:54.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:56:54.756 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:54.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:54 smithi171 ceph-mon[36033]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:56:54.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:54 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi049 2024-03-26T17:56:54.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:54 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi049 to 5696M 2024-03-26T17:56:54.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:56:54.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:56:54.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:55.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:55 smithi115 ceph-mon[36174]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:56:55.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:55 smithi049 ceph-mon[32331]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:56:55.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:55 smithi171 ceph-mon[36033]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:56:56.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:56 smithi115 ceph-mon[36174]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:56:56.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:56 smithi049 ceph-mon[32331]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:56:56.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:56 smithi171 ceph-mon[36033]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:56:57.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:57 smithi049 ceph-mon[32331]: from='client.? 172.21.15.115:0/2055422348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f64173b-87e4-4f35-a827-287345059695"}]: dispatch 2024-03-26T17:56:57.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:57 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f64173b-87e4-4f35-a827-287345059695"}]: dispatch 2024-03-26T17:56:57.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:57 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f64173b-87e4-4f35-a827-287345059695"}]': finished 2024-03-26T17:56:57.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:57 smithi049 ceph-mon[32331]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T17:56:57.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:56:57.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:57 smithi171 ceph-mon[36033]: from='client.? 172.21.15.115:0/2055422348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f64173b-87e4-4f35-a827-287345059695"}]: dispatch 2024-03-26T17:56:57.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:57 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f64173b-87e4-4f35-a827-287345059695"}]: dispatch 2024-03-26T17:56:57.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:57 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f64173b-87e4-4f35-a827-287345059695"}]': finished 2024-03-26T17:56:57.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:57 smithi171 ceph-mon[36033]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T17:56:57.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:56:57.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:57 smithi115 ceph-mon[36174]: from='client.? 172.21.15.115:0/2055422348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f64173b-87e4-4f35-a827-287345059695"}]: dispatch 2024-03-26T17:56:57.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:57 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f64173b-87e4-4f35-a827-287345059695"}]: dispatch 2024-03-26T17:56:57.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:57 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f64173b-87e4-4f35-a827-287345059695"}]': finished 2024-03-26T17:56:57.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:57 smithi115 ceph-mon[36174]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T17:56:57.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:56:58.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:58 smithi049 ceph-mon[32331]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:56:58.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:56:58 smithi049 ceph-mon[32331]: from='client.? 172.21.15.115:0/227295696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:56:58.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:58 smithi171 ceph-mon[36033]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:56:58.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:56:58 smithi171 ceph-mon[36033]: from='client.? 172.21.15.115:0/227295696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:56:58.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:58 smithi115 ceph-mon[36174]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:56:58.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:56:58 smithi115 ceph-mon[36174]: from='client.? 172.21.15.115:0/227295696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:57:00.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:00 smithi049 ceph-mon[32331]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:00.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:00 smithi171 ceph-mon[36033]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:00.862 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:00 smithi115 ceph-mon[36174]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:02.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:02 smithi049 ceph-mon[32331]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:02.761 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:02 smithi115 ceph-mon[36174]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:02.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:02 smithi171 ceph-mon[36033]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:04.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:04 smithi049 ceph-mon[32331]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:04.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:04 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T17:57:04.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:04 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:04.847 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:04 smithi115 ceph-mon[36174]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:04.847 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:04 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T17:57:04.847 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:04 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:04.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:04 smithi171 ceph-mon[36033]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:04.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:04 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T17:57:04.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:04 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:05.700 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:05 smithi115 ceph-mon[36174]: Deploying daemon osd.7 on smithi115 2024-03-26T17:57:05.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:05 smithi049 ceph-mon[32331]: Deploying daemon osd.7 on smithi115 2024-03-26T17:57:05.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:05 smithi171 ceph-mon[36033]: Deploying daemon osd.7 on smithi115 2024-03-26T17:57:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:06 smithi171 ceph-mon[36033]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:06.919 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:06 smithi115 ceph-mon[36174]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:07.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:06 smithi049 ceph-mon[32331]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:08.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:08 smithi171 ceph-mon[36033]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:08.899 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:08 smithi115 ceph-mon[36174]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:09.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:08 smithi049 ceph-mon[32331]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:10.660 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:10 smithi115 ceph-mon[36174]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:10.660 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:10 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:10.660 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:10 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:10.660 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:10 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:10.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:10 smithi171 ceph-mon[36033]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:10.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:10 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:10.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:10 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:10.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:10 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:11.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:10 smithi049 ceph-mon[32331]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:11.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:10 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:11.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:10 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:11.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:10 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:12.542 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:12 smithi115 ceph-mon[36174]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:12.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:12 smithi171 ceph-mon[36033]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:13.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:12 smithi049 ceph-mon[32331]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.660+0000 7fb3faffd700 1 -- 172.21.15.115:0/3940500582 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb4040627f0 con 0x7fb3f00a2fb0 2024-03-26T17:57:13.646 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 7 on host 'smithi115' 2024-03-26T17:57:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.662+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3940500582 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3f00a2fb0 msgr2=0x7fb3f00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.662+0000 7fb40bfe4700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3f00a2fb0 0x7fb3f00a5470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb400005950 tx=0x7fb4000058e0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.662+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3940500582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb404101570 msgr2=0x7fb4041985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.662+0000 7fb40bfe4700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb404101570 0x7fb4041985b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400d980 tx=0x7fb3f400dd40 comp rx=0 tx=0).stop 2024-03-26T17:57:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.662+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3940500582 shutdown_connections 2024-03-26T17:57:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.662+0000 7fb40bfe4700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3f00a2fb0 0x7fb3f00a5470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.662+0000 7fb40bfe4700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb404102910 0x7fb40419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.662+0000 7fb40bfe4700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb404101570 0x7fb4041985b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.662+0000 7fb40bfe4700 1 --2- 172.21.15.115:0/3940500582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb404101f30 0x7fb404198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.662+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3940500582 >> 172.21.15.115:0/3940500582 conn(0x7fb4040fc000 msgr2=0x7fb404105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.663+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3940500582 shutdown_connections 2024-03-26T17:57:13.649 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:57:13.663+0000 7fb40bfe4700 1 -- 172.21.15.115:0/3940500582 wait complete. 2024-03-26T17:57:14.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:14.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:14.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:14.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: from='osd.7 [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:57:14.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:57:14.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:13 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:14.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:14.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:14.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:14.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: from='osd.7 [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:57:14.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:57:14.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.256 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:13 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:14.283 DEBUG:teuthology.orchestra.run.smithi115:osd.7> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.7.service 2024-03-26T17:57:14.285 INFO:tasks.cephadm:Deploying osd.8 on smithi171 with /dev/vg_nvme/lv_2... 2024-03-26T17:57:14.285 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T17:57:14.315 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:14.315 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.315 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.316 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:14.316 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:14.316 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.316 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: from='osd.7 [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:57:14.316 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:57:14.316 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.316 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:14.316 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:13 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:14.532 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:57:15.732 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:15 smithi115 ceph-mon[36174]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T17:57:15.732 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:15 smithi115 ceph-mon[36174]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T17:57:15.732 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:15.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:15 smithi115 ceph-mon[36174]: from='osd.7 [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:57:15.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:15 smithi115 ceph-mon[36174]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:57:15.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:15 smithi049 ceph-mon[32331]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T17:57:15.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:15 smithi049 ceph-mon[32331]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T17:57:15.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:15.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:15 smithi049 ceph-mon[32331]: from='osd.7 [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:57:15.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:15 smithi049 ceph-mon[32331]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:57:15.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:15 smithi171 ceph-mon[36033]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T17:57:15.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:15 smithi171 ceph-mon[36033]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T17:57:15.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:15.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:15 smithi171 ceph-mon[36033]: from='osd.7 [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:57:15.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:15 smithi171 ceph-mon[36033]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:57:16.241 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 17:57:16 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T17:57:16.002+0000 7f07231f3700 -1 osd.7 0 waiting for initial osdmap 2024-03-26T17:57:16.241 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 17:57:16 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T17:57:16.012+0000 7f071afde700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:57:16.646 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T17:57:16.646 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T17:57:16.646 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-03-26T17:57:16.646 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-03-26T17:57:16.646 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126876 s, 826 MB/s 2024-03-26T17:57:16.646 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-03-26T17:57:16.840 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:16 smithi115 ceph-mon[36174]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:16.841 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:16 smithi115 ceph-mon[36174]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:57:16.841 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:16 smithi115 ceph-mon[36174]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T17:57:16.841 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:16 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:16.841 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:16 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:16.841 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:16 smithi115 ceph-mon[36174]: from='osd.7 ' entity='osd.7' 2024-03-26T17:57:16.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:16 smithi171 ceph-mon[36033]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:16.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:16 smithi171 ceph-mon[36033]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:57:16.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:16 smithi171 ceph-mon[36033]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T17:57:16.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:16 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:16.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:16 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:16.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:16 smithi171 ceph-mon[36033]: from='osd.7 ' entity='osd.7' 2024-03-26T17:57:17.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:16 smithi049 ceph-mon[32331]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:17.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:16 smithi049 ceph-mon[32331]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:57:17.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:16 smithi049 ceph-mon[32331]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T17:57:17.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:16 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:17.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:16 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:17.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:16 smithi049 ceph-mon[32331]: from='osd.7 ' entity='osd.7' 2024-03-26T17:57:17.298 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi171:vg_nvme/lv_2 2024-03-26T17:57:17.585 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:57:17.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:57:17.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:57:17.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:17.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: osd.7 [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] boot 2024-03-26T17:57:17.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T17:57:17.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:17.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:17.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:17.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:17.831 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:17.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:17.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:17.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:17.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:17.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:17.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:17 smithi171 ceph-mon[36033]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T17:57:17.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:57:17.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:57:17.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:17.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: osd.7 [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] boot 2024-03-26T17:57:17.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T17:57:17.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:17.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:17.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:17.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:17.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:17.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:17.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:17.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:17.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:17.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:17.991 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:17 smithi115 ceph-mon[36174]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: osd.7 [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] boot 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:18.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:18.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:18.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:18.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:18.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:18.006 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:17 smithi049 ceph-mon[32331]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T17:57:18.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:18 smithi171 ceph-mon[36033]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:18.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:18 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi115 2024-03-26T17:57:18.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:18 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi115 to 5693M 2024-03-26T17:57:18.695 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:18 smithi171 ceph-mon[36033]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T17:57:18.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:18 smithi115 ceph-mon[36174]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:18.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:18 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi115 2024-03-26T17:57:18.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:18 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi115 to 5693M 2024-03-26T17:57:18.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:18 smithi115 ceph-mon[36174]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T17:57:19.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:18 smithi049 ceph-mon[32331]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:57:19.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:18 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi115 2024-03-26T17:57:19.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:18 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi115 to 5693M 2024-03-26T17:57:19.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:18 smithi049 ceph-mon[32331]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T17:57:19.235 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.246+0000 7fd72ecc6700 1 -- 172.21.15.171:0/3934623984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd728102ed0 msgr2=0x7fd728109780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:19.235 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.246+0000 7fd72ecc6700 1 --2- 172.21.15.171:0/3934623984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd728102ed0 0x7fd728109780 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd72000b0a0 tx=0x7fd72000b3b0 comp rx=0 tx=0).stop 2024-03-26T17:57:19.235 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.246+0000 7fd72ecc6700 1 -- 172.21.15.171:0/3934623984 shutdown_connections 2024-03-26T17:57:19.235 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.246+0000 7fd72ecc6700 1 --2- 172.21.15.171:0/3934623984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd728102ed0 0x7fd728109780 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:19.235 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 --2- 172.21.15.171:0/3934623984 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728101330 0x7fd728101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 --2- 172.21.15.171:0/3934623984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd728102530 0x7fd728102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 -- 172.21.15.171:0/3934623984 >> 172.21.15.171:0/3934623984 conn(0x7fd7280fc8b0 msgr2=0x7fd7280fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 -- 172.21.15.171:0/3934623984 shutdown_connections 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 -- 172.21.15.171:0/3934623984 wait complete. 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 Processor -- start 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 -- start start 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728101330 0x7fd7281052c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd728102530 0x7fd728103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd728102ed0 0x7fd728103e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd72810cf70 con 0x7fd728101330 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd72810cdf0 con 0x7fd728102ed0 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd72ecc6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd72810d0f0 con 0x7fd728102530 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd727fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd728102530 0x7fd728103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:19.236 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd727fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd728102530 0x7fd728103910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:51972/0 (socket says 172.21.15.171:51972) 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.247+0000 7fd727fff700 1 -- 172.21.15.171:0/1744928027 learned_addr learned my addr 172.21.15.171:0/1744928027 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd72ca62700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728101330 0x7fd7281052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd727fff700 1 -- 172.21.15.171:0/1744928027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd728102ed0 msgr2=0x7fd728103e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd727fff700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd728102ed0 0x7fd728103e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd727fff700 1 -- 172.21.15.171:0/1744928027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728101330 msgr2=0x7fd7281052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd727fff700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728101330 0x7fd7281052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd727fff700 1 -- 172.21.15.171:0/1744928027 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd728104700 con 0x7fd728102530 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd727fff700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd728102530 0x7fd728103910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c970 tx=0x7fd71400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd725ffb700 1 -- 172.21.15.171:0/1744928027 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714007aa0 con 0x7fd728102530 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd72ecc6700 1 -- 172.21.15.171:0/1744928027 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7281a26f0 con 0x7fd728102530 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd725ffb700 1 -- 172.21.15.171:0/1744928027 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd714004d10 con 0x7fd728102530 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd725ffb700 1 -- 172.21.15.171:0/1744928027 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71400f450 con 0x7fd728102530 2024-03-26T17:57:19.237 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.248+0000 7fd72ecc6700 1 -- 172.21.15.171:0/1744928027 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7281a2bb0 con 0x7fd728102530 2024-03-26T17:57:19.238 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.249+0000 7fd72ecc6700 1 -- 172.21.15.171:0/1744928027 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd728069070 con 0x7fd728102530 2024-03-26T17:57:19.238 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.250+0000 7fd725ffb700 1 -- 172.21.15.171:0/1744928027 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fd714004750 con 0x7fd728102530 2024-03-26T17:57:19.238 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.251+0000 7fd725ffb700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7180a2fb0 0x7fd7180a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:19.238 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.251+0000 7fd725ffb700 1 -- 172.21.15.171:0/1744928027 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7fd7140ce6c0 con 0x7fd728102530 2024-03-26T17:57:19.238 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.252+0000 7fd72ca62700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7180a2fb0 0x7fd7180a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:19.239 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.252+0000 7fd72ca62700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7180a2fb0 0x7fd7180a5470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd728102390 tx=0x7fd71c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:19.242 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.255+0000 7fd725ffb700 1 -- 172.21.15.171:0/1744928027 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd71409c0c0 con 0x7fd728102530 2024-03-26T17:57:19.445 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:19.459+0000 7fd72ecc6700 1 -- 172.21.15.171:0/1744928027 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd7280fe8f0 con 0x7fd7180a2fb0 2024-03-26T17:57:19.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:57:19.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:57:19.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:19.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:57:19.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:57:19.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:20.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:57:20.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:57:20.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:20.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:20 smithi171 ceph-mon[36033]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:20.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:20 smithi171 ceph-mon[36033]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:57:20.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:20 smithi171 ceph-mon[36033]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T17:57:20.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:20 smithi115 ceph-mon[36174]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:20.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:20 smithi115 ceph-mon[36174]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:57:20.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:20 smithi115 ceph-mon[36174]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T17:57:21.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:20 smithi049 ceph-mon[32331]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:21.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:20 smithi049 ceph-mon[32331]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:57:21.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:20 smithi049 ceph-mon[32331]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T17:57:21.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:21 smithi171 ceph-mon[36033]: osdmap e56: 8 total, 8 up, 8 in 2024-03-26T17:57:21.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:21 smithi115 ceph-mon[36174]: osdmap e56: 8 total, 8 up, 8 in 2024-03-26T17:57:22.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:21 smithi049 ceph-mon[32331]: osdmap e56: 8 total, 8 up, 8 in 2024-03-26T17:57:22.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:22 smithi171 ceph-mon[36033]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:22.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:22 smithi171 ceph-mon[36033]: from='client.? 172.21.15.171:0/1102541320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c2ba0d-b239-4b21-9ecf-ab1a4d948977"}]: dispatch 2024-03-26T17:57:22.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:22 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c2ba0d-b239-4b21-9ecf-ab1a4d948977"}]: dispatch 2024-03-26T17:57:22.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:22 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1c2ba0d-b239-4b21-9ecf-ab1a4d948977"}]': finished 2024-03-26T17:57:22.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:22 smithi171 ceph-mon[36033]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T17:57:22.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:22 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:22.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:22 smithi115 ceph-mon[36174]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:22.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:22 smithi115 ceph-mon[36174]: from='client.? 172.21.15.171:0/1102541320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c2ba0d-b239-4b21-9ecf-ab1a4d948977"}]: dispatch 2024-03-26T17:57:22.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:22 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c2ba0d-b239-4b21-9ecf-ab1a4d948977"}]: dispatch 2024-03-26T17:57:22.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:22 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1c2ba0d-b239-4b21-9ecf-ab1a4d948977"}]': finished 2024-03-26T17:57:22.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:22 smithi115 ceph-mon[36174]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T17:57:22.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:22 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:23.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:22 smithi049 ceph-mon[32331]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:23.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:22 smithi049 ceph-mon[32331]: from='client.? 172.21.15.171:0/1102541320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c2ba0d-b239-4b21-9ecf-ab1a4d948977"}]: dispatch 2024-03-26T17:57:23.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:22 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c2ba0d-b239-4b21-9ecf-ab1a4d948977"}]: dispatch 2024-03-26T17:57:23.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:22 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1c2ba0d-b239-4b21-9ecf-ab1a4d948977"}]': finished 2024-03-26T17:57:23.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:22 smithi049 ceph-mon[32331]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T17:57:23.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:22 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:23.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:23 smithi171 ceph-mon[36033]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:23.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:23 smithi171 ceph-mon[36033]: from='client.? 172.21.15.171:0/3459701624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:57:23.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:23 smithi115 ceph-mon[36174]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:23.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:23 smithi115 ceph-mon[36174]: from='client.? 172.21.15.171:0/3459701624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:57:24.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:23 smithi049 ceph-mon[32331]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:24.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:23 smithi049 ceph-mon[32331]: from='client.? 172.21.15.171:0/3459701624' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:57:25.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:25 smithi115 ceph-mon[36174]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:26.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:25 smithi049 ceph-mon[32331]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:26.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:25 smithi171 ceph-mon[36033]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:27.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:27 smithi115 ceph-mon[36174]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:28.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:27 smithi049 ceph-mon[32331]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:28.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:27 smithi171 ceph-mon[36033]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:29.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:29 smithi171 ceph-mon[36033]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:29.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:29 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T17:57:29.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:29 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:29.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:29 smithi171 ceph-mon[36033]: Deploying daemon osd.8 on smithi171 2024-03-26T17:57:29.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:29 smithi115 ceph-mon[36174]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:29.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:29 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T17:57:29.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:29 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:29.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:29 smithi115 ceph-mon[36174]: Deploying daemon osd.8 on smithi171 2024-03-26T17:57:30.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:29 smithi049 ceph-mon[32331]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:30.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:29 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T17:57:30.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:29 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:30.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:29 smithi049 ceph-mon[32331]: Deploying daemon osd.8 on smithi171 2024-03-26T17:57:31.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:31 smithi115 ceph-mon[36174]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:32.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:31 smithi049 ceph-mon[32331]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:32.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:31 smithi171 ceph-mon[36033]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:33.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:33 smithi115 ceph-mon[36174]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:34.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:33 smithi049 ceph-mon[32331]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:34.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:33 smithi171 ceph-mon[36033]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:36.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:36 smithi171 ceph-mon[36033]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:36.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:36.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:36.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:36.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:36 smithi115 ceph-mon[36174]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:36.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:36.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:36.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:36.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:36 smithi049 ceph-mon[32331]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:36.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:36.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:36.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:38.213 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:38 smithi171 ceph-mon[36033]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:38.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:38 smithi115 ceph-mon[36174]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:38.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:38 smithi049 ceph-mon[32331]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:38.511 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 8 on host 'smithi171' 2024-03-26T17:57:38.511 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.526+0000 7fd725ffb700 1 -- 172.21.15.171:0/1744928027 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd7280fe8f0 con 0x7fd7180a2fb0 2024-03-26T17:57:38.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 -- 172.21.15.171:0/1744928027 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7180a2fb0 msgr2=0x7fd7180a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:38.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7180a2fb0 0x7fd7180a5470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd728102390 tx=0x7fd71c00a3b0 comp rx=0 tx=0).stop 2024-03-26T17:57:38.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 -- 172.21.15.171:0/1744928027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd728102530 msgr2=0x7fd728103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:38.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd728102530 0x7fd728103910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c970 tx=0x7fd71400cc80 comp rx=0 tx=0).stop 2024-03-26T17:57:38.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 -- 172.21.15.171:0/1744928027 shutdown_connections 2024-03-26T17:57:38.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7180a2fb0 0x7fd7180a5470 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:38.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd728102ed0 0x7fd728103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:38.513 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd728101330 0x7fd7281052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:38.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 --2- 172.21.15.171:0/1744928027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd728102530 0x7fd728103910 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:38.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 -- 172.21.15.171:0/1744928027 >> 172.21.15.171:0/1744928027 conn(0x7fd7280fc8b0 msgr2=0x7fd7280fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:38.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 -- 172.21.15.171:0/1744928027 shutdown_connections 2024-03-26T17:57:38.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:57:38.528+0000 7fd72ecc6700 1 -- 172.21.15.171:0/1744928027 wait complete. 2024-03-26T17:57:39.177 DEBUG:teuthology.orchestra.run.smithi171:osd.8> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.8.service 2024-03-26T17:57:39.179 INFO:tasks.cephadm:Deploying osd.9 on smithi049 with /dev/vg_nvme/lv_1... 2024-03-26T17:57:39.180 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T17:57:39.444 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:57:39.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:39 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:39 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:39 smithi049 ceph-mon[32331]: from='osd.8 [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:57:39.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:39 smithi049 ceph-mon[32331]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:57:39.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:39 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:39 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:39 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:39 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:39 smithi171 ceph-mon[36033]: from='osd.8 [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:57:39.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:39 smithi171 ceph-mon[36033]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:57:39.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:39 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:39 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:39 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:39 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:39 smithi115 ceph-mon[36174]: from='osd.8 [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:57:39.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:39 smithi115 ceph-mon[36174]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:57:39.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:39 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:39.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:39 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:40.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:40 smithi171 ceph-mon[36033]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:40.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:40 smithi171 ceph-mon[36033]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T17:57:40.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:40 smithi171 ceph-mon[36033]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T17:57:40.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:40 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:40.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:40 smithi171 ceph-mon[36033]: from='osd.8 [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:57:40.517 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:40 smithi171 ceph-mon[36033]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:57:40.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:40 smithi049 ceph-mon[32331]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:40.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:40 smithi049 ceph-mon[32331]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T17:57:40.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:40 smithi049 ceph-mon[32331]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T17:57:40.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:40 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:40.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:40 smithi049 ceph-mon[32331]: from='osd.8 [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:57:40.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:40 smithi049 ceph-mon[32331]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:57:40.859 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 17:57:40 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T17:57:40.531+0000 7f35b3cb5700 -1 osd.8 0 waiting for initial osdmap 2024-03-26T17:57:40.859 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 17:57:40 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T17:57:40.541+0000 7f35afaa8700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:57:40.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:40 smithi115 ceph-mon[36174]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:40.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:40 smithi115 ceph-mon[36174]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T17:57:40.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:40 smithi115 ceph-mon[36174]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T17:57:40.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:40 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:40.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:40 smithi115 ceph-mon[36174]: from='osd.8 [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:57:40.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:40 smithi115 ceph-mon[36174]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:57:41.515 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:41 smithi171 ceph-mon[36033]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:57:41.515 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:41 smithi171 ceph-mon[36033]: osdmap e59: 9 total, 8 up, 9 in 2024-03-26T17:57:41.515 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:41 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:41.515 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:41 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T17:57:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T17:57:41.717 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2024-03-26T17:57:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2024-03-26T17:57:41.717 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00842466 s, 1.2 GB/s 2024-03-26T17:57:41.717 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2024-03-26T17:57:41.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:41 smithi115 ceph-mon[36174]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:57:41.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:41 smithi115 ceph-mon[36174]: osdmap e59: 9 total, 8 up, 9 in 2024-03-26T17:57:41.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:41 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:41.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:41 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:42.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:41 smithi049 ceph-mon[32331]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:57:42.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:41 smithi049 ceph-mon[32331]: osdmap e59: 9 total, 8 up, 9 in 2024-03-26T17:57:42.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:41 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:42.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:41 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:42.398 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi049:vg_nvme/lv_1 2024-03-26T17:57:42.651 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:57:42.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:42 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:57:42.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:42 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:57:42.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:42 smithi171 ceph-mon[36033]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:42.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:42 smithi171 ceph-mon[36033]: osd.8 [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] boot 2024-03-26T17:57:42.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:42 smithi171 ceph-mon[36033]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T17:57:42.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:42 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:42.752 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:42 smithi171 ceph-mon[36033]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T17:57:42.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:42 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:57:42.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:42 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:57:42.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:42 smithi049 ceph-mon[32331]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:42.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:42 smithi049 ceph-mon[32331]: osd.8 [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] boot 2024-03-26T17:57:42.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:42 smithi049 ceph-mon[32331]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T17:57:42.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:42 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:42.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:42 smithi049 ceph-mon[32331]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T17:57:42.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:42 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:57:42.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:42 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:57:42.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:42 smithi115 ceph-mon[36174]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:57:42.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:42 smithi115 ceph-mon[36174]: osd.8 [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] boot 2024-03-26T17:57:42.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:42 smithi115 ceph-mon[36174]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T17:57:42.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:42 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:57:42.990 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:42 smithi115 ceph-mon[36174]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T17:57:44.074 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:57:44.074 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi171 2024-03-26T17:57:44.074 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.074 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.074 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi171 to 5693M 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.075 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:43 smithi049 ceph-mon[32331]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T17:57:44.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:57:44.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi171 2024-03-26T17:57:44.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi171 to 5693M 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.110 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:43 smithi171 ceph-mon[36033]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi171 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi171 to 5693M 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:44.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:44.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:44.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:44.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:44.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:57:44.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:43 smithi115 ceph-mon[36174]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T17:57:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.556+0000 7f26da4c6700 1 -- 172.21.15.49:0/1042423220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d4102ed0 msgr2=0x7f26d41052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.556+0000 7f26da4c6700 1 --2- 172.21.15.49:0/1042423220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d4102ed0 0x7f26d41052c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009990 tx=0x7f26c4009ca0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.556+0000 7f26da4c6700 1 -- 172.21.15.49:0/1042423220 shutdown_connections 2024-03-26T17:57:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.556+0000 7f26da4c6700 1 --2- 172.21.15.49:0/1042423220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d4108130 0x7f26d410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.556+0000 7f26da4c6700 1 --2- 172.21.15.49:0/1042423220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d4102ed0 0x7f26d41052c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.556+0000 7f26da4c6700 1 --2- 172.21.15.49:0/1042423220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d4105800 0x7f26d4107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.556+0000 7f26da4c6700 1 -- 172.21.15.49:0/1042423220 >> 172.21.15.49:0/1042423220 conn(0x7f26d40fc880 msgr2=0x7f26d40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.556+0000 7f26da4c6700 1 -- 172.21.15.49:0/1042423220 shutdown_connections 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.556+0000 7f26da4c6700 1 -- 172.21.15.49:0/1042423220 wait complete. 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.557+0000 7f26da4c6700 1 Processor -- start 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.557+0000 7f26da4c6700 1 -- start start 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.557+0000 7f26da4c6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d4102ed0 0x7f26d419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.557+0000 7f26da4c6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d4105800 0x7f26d419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.557+0000 7f26da4c6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d4108130 0x7f26d41a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.557+0000 7f26da4c6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f26d410dff0 con 0x7f26d4102ed0 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.557+0000 7f26da4c6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f26d410de70 con 0x7f26d4108130 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.557+0000 7f26da4c6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f26d410e170 con 0x7f26d4105800 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d4102ed0 0x7f26d419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d4102ed0 0x7f26d419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38376/0 (socket says 172.21.15.49:38376) 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d3fff700 1 -- 172.21.15.49:0/978459669 learned_addr learned my addr 172.21.15.49:0/978459669 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d8a63700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d4108130 0x7f26d41a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d37fe700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d4105800 0x7f26d419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d3fff700 1 -- 172.21.15.49:0/978459669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d4105800 msgr2=0x7f26d419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d3fff700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d4105800 0x7f26d419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d3fff700 1 -- 172.21.15.49:0/978459669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d4108130 msgr2=0x7f26d41a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d3fff700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d4108130 0x7f26d41a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d3fff700 1 -- 172.21.15.49:0/978459669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d41a2470 con 0x7f26d4102ed0 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d8a63700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d4108130 0x7f26d41a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d37fe700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d4105800 0x7f26d419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d3fff700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d4102ed0 0x7f26d419a810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26c4004790 tx=0x7f26c4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d17fa700 1 -- 172.21.15.49:0/978459669 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4027070 con 0x7f26d4102ed0 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26da4c6700 1 -- 172.21.15.49:0/978459669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d41a2700 con 0x7f26d4102ed0 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26da4c6700 1 -- 172.21.15.49:0/978459669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d41a2bb0 con 0x7f26d4102ed0 2024-03-26T17:57:44.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d17fa700 1 -- 172.21.15.49:0/978459669 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c4004210 con 0x7f26d4102ed0 2024-03-26T17:57:44.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.558+0000 7f26d17fa700 1 -- 172.21.15.49:0/978459669 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c400f460 con 0x7f26d4102ed0 2024-03-26T17:57:44.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.559+0000 7f26da4c6700 1 -- 172.21.15.49:0/978459669 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d4069070 con 0x7f26d4102ed0 2024-03-26T17:57:44.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.560+0000 7f26d17fa700 1 -- 172.21.15.49:0/978459669 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f26c402a070 con 0x7f26d4102ed0 2024-03-26T17:57:44.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.561+0000 7f26d17fa700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f26bc0a2fb0 0x7f26bc0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.561+0000 7f26d37fe700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f26bc0a2fb0 0x7f26bc0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.562+0000 7f26d17fa700 1 -- 172.21.15.49:0/978459669 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7f26c40cfd50 con 0x7f26d4102ed0 2024-03-26T17:57:44.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.562+0000 7f26d37fe700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f26bc0a2fb0 0x7f26bc0a5470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f26c8005fd0 tx=0x7f26c8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.565+0000 7f26d17fa700 1 -- 172.21.15.49:0/978459669 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f26c4013070 con 0x7f26d4102ed0 2024-03-26T17:57:44.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:57:44.781+0000 7f26da4c6700 1 -- 172.21.15.49:0/978459669 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f26d40627f0 con 0x7f26bc0a2fb0 2024-03-26T17:57:45.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:57:45.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:57:45.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:45.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:57:45.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:57:45.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:45.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:57:45.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:57:45.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:46.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:45 smithi049 ceph-mon[32331]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:57:46.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:45 smithi049 ceph-mon[32331]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:57:46.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:45 smithi049 ceph-mon[32331]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T17:57:46.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:45 smithi171 ceph-mon[36033]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:57:46.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:45 smithi171 ceph-mon[36033]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:57:46.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:45 smithi171 ceph-mon[36033]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T17:57:46.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:45 smithi115 ceph-mon[36174]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:57:46.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:45 smithi115 ceph-mon[36174]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:57:46.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:45 smithi115 ceph-mon[36174]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T17:57:46.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:46 smithi049 ceph-mon[32331]: osdmap e64: 9 total, 9 up, 9 in 2024-03-26T17:57:47.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:46 smithi171 ceph-mon[36033]: osdmap e64: 9 total, 9 up, 9 in 2024-03-26T17:57:47.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:46 smithi115 ceph-mon[36174]: osdmap e64: 9 total, 9 up, 9 in 2024-03-26T17:57:48.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:47 smithi171 ceph-mon[36033]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:57:48.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:47 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2342107944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0c8e333-fbec-4186-90aa-a713043370e3"}]: dispatch 2024-03-26T17:57:48.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:47 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2342107944' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0c8e333-fbec-4186-90aa-a713043370e3"}]': finished 2024-03-26T17:57:48.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:47 smithi171 ceph-mon[36033]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T17:57:48.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:57:48.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:47 smithi115 ceph-mon[36174]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:57:48.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:47 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2342107944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0c8e333-fbec-4186-90aa-a713043370e3"}]: dispatch 2024-03-26T17:57:48.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:47 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2342107944' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0c8e333-fbec-4186-90aa-a713043370e3"}]': finished 2024-03-26T17:57:48.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:47 smithi115 ceph-mon[36174]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T17:57:48.199 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:57:48.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:47 smithi049 ceph-mon[32331]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:57:48.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:47 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2342107944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0c8e333-fbec-4186-90aa-a713043370e3"}]: dispatch 2024-03-26T17:57:48.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:47 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2342107944' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0c8e333-fbec-4186-90aa-a713043370e3"}]': finished 2024-03-26T17:57:48.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:47 smithi049 ceph-mon[32331]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T17:57:48.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:57:49.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:48 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1766938869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:57:49.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:48 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1766938869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:57:49.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:48 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1766938869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:57:50.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:49 smithi171 ceph-mon[36033]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-26T17:57:50.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:49 smithi115 ceph-mon[36174]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-26T17:57:50.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:49 smithi049 ceph-mon[32331]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-26T17:57:52.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:51 smithi049 ceph-mon[32331]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:57:52.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:51 smithi171 ceph-mon[36033]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:57:52.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:51 smithi115 ceph-mon[36174]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:57:54.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:53 smithi171 ceph-mon[36033]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-26T17:57:54.191 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:53 smithi049 ceph-mon[32331]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-26T17:57:54.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:53 smithi115 ceph-mon[36174]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-26T17:57:56.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:55 smithi171 ceph-mon[36033]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-26T17:57:56.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:55 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T17:57:56.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:55 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:56.226 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:55 smithi049 ceph-mon[32331]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-26T17:57:56.226 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:55 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T17:57:56.226 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:55 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:56.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:55 smithi115 ceph-mon[36174]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-26T17:57:56.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:55 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T17:57:56.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:55 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:57.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:56 smithi171 ceph-mon[36033]: Deploying daemon osd.9 on smithi049 2024-03-26T17:57:57.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:56 smithi115 ceph-mon[36174]: Deploying daemon osd.9 on smithi049 2024-03-26T17:57:57.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:56 smithi049 ceph-mon[32331]: Deploying daemon osd.9 on smithi049 2024-03-26T17:57:58.074 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:57 smithi049 ceph-mon[32331]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:57:58.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:57 smithi171 ceph-mon[36033]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:57:58.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:57 smithi115 ceph-mon[36174]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:58:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:57:59 smithi171 ceph-mon[36033]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:00.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:57:59 smithi115 ceph-mon[36174]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:00.246 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:57:59 smithi049 ceph-mon[32331]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:01 smithi171 ceph-mon[36033]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:02.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:02.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:01 smithi115 ceph-mon[36174]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:02.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:02.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:02.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:02.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:01 smithi049 ceph-mon[32331]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:02.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:02.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:02.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:04.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:03 smithi049 ceph-mon[32331]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:04.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:03 smithi171 ceph-mon[36033]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:04.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:03 smithi115 ceph-mon[36174]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:05.789 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 9 on host 'smithi049' 2024-03-26T17:58:05.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.805+0000 7f26d17fa700 1 -- 172.21.15.49:0/978459669 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f26d40627f0 con 0x7f26bc0a2fb0 2024-03-26T17:58:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 -- 172.21.15.49:0/978459669 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f26bc0a2fb0 msgr2=0x7f26bc0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f26bc0a2fb0 0x7f26bc0a5470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f26c8005fd0 tx=0x7f26c8005ee0 comp rx=0 tx=0).stop 2024-03-26T17:58:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 -- 172.21.15.49:0/978459669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d4102ed0 msgr2=0x7f26d419a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d4102ed0 0x7f26d419a810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26c4004790 tx=0x7f26c4009580 comp rx=0 tx=0).stop 2024-03-26T17:58:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 -- 172.21.15.49:0/978459669 shutdown_connections 2024-03-26T17:58:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f26bc0a2fb0 0x7f26bc0a5470 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d4108130 0x7f26d41a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26d4102ed0 0x7f26d419a810 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:05.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 --2- 172.21.15.49:0/978459669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26d4105800 0x7f26d419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 -- 172.21.15.49:0/978459669 >> 172.21.15.49:0/978459669 conn(0x7f26d40fc880 msgr2=0x7f26d4102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 -- 172.21.15.49:0/978459669 shutdown_connections 2024-03-26T17:58:05.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:05.807+0000 7f26da4c6700 1 -- 172.21.15.49:0/978459669 wait complete. 2024-03-26T17:58:06.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:05 smithi049 ceph-mon[32331]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:06.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:06.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:06.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:05 smithi049 ceph-mon[32331]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T17:58:06.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.005 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:05 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:06.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:05 smithi171 ceph-mon[36033]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:06.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:06.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:06.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:05 smithi171 ceph-mon[36033]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T17:58:06.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.189 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:05 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:06.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:05 smithi115 ceph-mon[36174]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:06.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:06.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:06.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:05 smithi115 ceph-mon[36174]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T17:58:06.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:06.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:05 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:06.597 DEBUG:teuthology.orchestra.run.smithi049:osd.9> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.9.service 2024-03-26T17:58:06.600 INFO:tasks.cephadm:Deploying osd.10 on smithi115 with /dev/vg_nvme/lv_1... 2024-03-26T17:58:06.600 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T17:58:06.842 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:58:07.078 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:06 smithi049 ceph-mon[32331]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T17:58:07.078 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:06 smithi049 ceph-mon[32331]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T17:58:07.079 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:06 smithi049 ceph-mon[32331]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:58:07.079 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:06 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:07.079 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 17:58:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T17:58:06.919+0000 7f1129127700 -1 osd.9 0 waiting for initial osdmap 2024-03-26T17:58:07.079 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 17:58:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T17:58:06.931+0000 7f1124f1a700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:58:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:06 smithi115 ceph-mon[36174]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T17:58:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:06 smithi115 ceph-mon[36174]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T17:58:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:06 smithi115 ceph-mon[36174]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:58:07.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:06 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:07.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:06 smithi171 ceph-mon[36033]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T17:58:07.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:06 smithi171 ceph-mon[36033]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T17:58:07.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:06 smithi171 ceph-mon[36033]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2024-03-26T17:58:07.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:06 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:08.066 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:07 smithi049 ceph-mon[32331]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:08.066 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:07 smithi049 ceph-mon[32331]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:58:08.067 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:07 smithi049 ceph-mon[32331]: osdmap e67: 10 total, 9 up, 10 in 2024-03-26T17:58:08.067 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:07 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:08.067 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:07 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:08.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:07 smithi115 ceph-mon[36174]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:08.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:07 smithi115 ceph-mon[36174]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:58:08.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:07 smithi115 ceph-mon[36174]: osdmap e67: 10 total, 9 up, 10 in 2024-03-26T17:58:08.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:07 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:08.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:07 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:08.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:07 smithi171 ceph-mon[36033]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:08.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:07 smithi171 ceph-mon[36033]: from='osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2024-03-26T17:58:08.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:07 smithi171 ceph-mon[36033]: osdmap e67: 10 total, 9 up, 10 in 2024-03-26T17:58:08.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:07 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:08.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:07 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:08.829 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T17:58:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T17:58:08.830 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-03-26T17:58:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-03-26T17:58:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011176 s, 938 MB/s 2024-03-26T17:58:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-03-26T17:58:09.149 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:08 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:58:09.149 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:08 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:58:09.150 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:08 smithi115 ceph-mon[36174]: osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] boot 2024-03-26T17:58:09.150 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:08 smithi115 ceph-mon[36174]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T17:58:09.150 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:08 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:09.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:08 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:58:09.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:08 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:58:09.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:08 smithi049 ceph-mon[32331]: osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] boot 2024-03-26T17:58:09.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:08 smithi049 ceph-mon[32331]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T17:58:09.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:08 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:09.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:08 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:58:09.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:08 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:58:09.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:08 smithi171 ceph-mon[36033]: osd.9 [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] boot 2024-03-26T17:58:09.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:08 smithi171 ceph-mon[36033]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T17:58:09.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:08 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:58:09.588 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_1 2024-03-26T17:58:09.828 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:58:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: osdmap e69: 10 total, 10 up, 10 in 2024-03-26T17:58:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.196 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:10.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:10.197 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:10.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: osdmap e69: 10 total, 10 up, 10 in 2024-03-26T17:58:10.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:10.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:10.255 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:58:10.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: osdmap e69: 10 total, 10 up, 10 in 2024-03-26T17:58:10.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:10.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:10.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:10.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:10.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:11.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:10 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi049 2024-03-26T17:58:11.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:10 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi049 to 4272M 2024-03-26T17:58:11.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:10 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi049 2024-03-26T17:58:11.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:10 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi049 to 4272M 2024-03-26T17:58:11.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:10 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi049 2024-03-26T17:58:11.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:10 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi049 to 4272M 2024-03-26T17:58:11.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 -- 172.21.15.115:0/3003162305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effc0105800 msgr2=0x7effc0107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:11.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 --2- 172.21.15.115:0/3003162305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effc0105800 0x7effc0107bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7effac009990 tx=0x7effac009ca0 comp rx=0 tx=0).stop 2024-03-26T17:58:11.550 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 -- 172.21.15.115:0/3003162305 shutdown_connections 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 --2- 172.21.15.115:0/3003162305 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc0108130 0x7effc010a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 --2- 172.21.15.115:0/3003162305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0102ed0 0x7effc01052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 --2- 172.21.15.115:0/3003162305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effc0105800 0x7effc0107bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 -- 172.21.15.115:0/3003162305 >> 172.21.15.115:0/3003162305 conn(0x7effc00fc8a0 msgr2=0x7effc00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 -- 172.21.15.115:0/3003162305 shutdown_connections 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 -- 172.21.15.115:0/3003162305 wait complete. 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 Processor -- start 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 -- start start 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effc0102ed0 0x7effc0102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.564+0000 7effc6fe0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0105800 0x7effc0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc6fe0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc0108130 0x7effc01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc6fe0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7effc010e030 con 0x7effc0108130 2024-03-26T17:58:11.551 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc6fe0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7effc010deb0 con 0x7effc0102ed0 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc6fe0700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7effc010e1b0 con 0x7effc0105800 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc4d7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effc0102ed0 0x7effc0102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc4d7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effc0102ed0 0x7effc0102830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39158/0 (socket says 172.21.15.115:39158) 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc4d7c700 1 -- 172.21.15.115:0/13399834 learned_addr learned my addr 172.21.15.115:0/13399834 (peer_addr_for_me v2:172.21.15.115:0/0) 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effbffff700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0105800 0x7effc0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc557d700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc0108130 0x7effc01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc4d7c700 1 -- 172.21.15.115:0/13399834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0105800 msgr2=0x7effc0100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc4d7c700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0105800 0x7effc0100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc4d7c700 1 -- 172.21.15.115:0/13399834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc0108130 msgr2=0x7effc01013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc4d7c700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc0108130 0x7effc01013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc4d7c700 1 -- 172.21.15.115:0/13399834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc0101c70 con 0x7effc0102ed0 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc4d7c700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effc0102ed0 0x7effc0102830 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7effb400b530 tx=0x7effb400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effbdffb700 1 -- 172.21.15.115:0/13399834 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4015020 con 0x7effc0102ed0 2024-03-26T17:58:11.552 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effbdffb700 1 -- 172.21.15.115:0/13399834 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb4011d10 con 0x7effc0102ed0 2024-03-26T17:58:11.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effc6fe0700 1 -- 172.21.15.115:0/13399834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc01a2830 con 0x7effc0102ed0 2024-03-26T17:58:11.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.565+0000 7effbdffb700 1 -- 172.21.15.115:0/13399834 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb400f2e0 con 0x7effc0102ed0 2024-03-26T17:58:11.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.566+0000 7effc6fe0700 1 -- 172.21.15.115:0/13399834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc01a2df0 con 0x7effc0102ed0 2024-03-26T17:58:11.553 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.567+0000 7effc6fe0700 1 -- 172.21.15.115:0/13399834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc0069070 con 0x7effc0102ed0 2024-03-26T17:58:11.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.568+0000 7effbdffb700 1 -- 172.21.15.115:0/13399834 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7effb4018020 con 0x7effc0102ed0 2024-03-26T17:58:11.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.568+0000 7effbdffb700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7effb00a3030 0x7effb00a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:11.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.569+0000 7effbdffb700 1 -- 172.21.15.115:0/13399834 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7effb40ce210 con 0x7effc0102ed0 2024-03-26T17:58:11.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.569+0000 7effbffff700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7effb00a3030 0x7effb00a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:11.554 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.569+0000 7effbffff700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7effb00a3030 0x7effb00a54f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7effac00b5c0 tx=0x7effac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:11.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.573+0000 7effbdffb700 1 -- 172.21.15.115:0/13399834 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7effb401f020 con 0x7effc0102ed0 2024-03-26T17:58:11.762 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:11.777+0000 7effc6fe0700 1 -- 172.21.15.115:0/13399834 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7effc00627f0 con 0x7effb00a3030 2024-03-26T17:58:12.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:11 smithi115 ceph-mon[36174]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:12.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:11 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:58:12.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:11 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:58:12.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:11 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:12.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:11 smithi049 ceph-mon[32331]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:12.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:58:12.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:58:12.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:11 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:12.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:11 smithi171 ceph-mon[36033]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:12.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:11 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:58:12.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:11 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:58:12.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:11 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:13.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:12 smithi115 ceph-mon[36174]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:58:13.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:12 smithi049 ceph-mon[32331]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:58:13.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:12 smithi171 ceph-mon[36033]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:58:14.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:13 smithi115 ceph-mon[36174]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:14.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:13 smithi049 ceph-mon[32331]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:14.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:13 smithi171 ceph-mon[36033]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:15.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:14 smithi115 ceph-mon[36174]: from='client.? 172.21.15.115:0/3350434647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4828303-f823-4e3c-8298-33bba2eda98d"}]: dispatch 2024-03-26T17:58:15.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:14 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4828303-f823-4e3c-8298-33bba2eda98d"}]: dispatch 2024-03-26T17:58:15.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:14 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4828303-f823-4e3c-8298-33bba2eda98d"}]': finished 2024-03-26T17:58:15.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:14 smithi115 ceph-mon[36174]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T17:58:15.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:14 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:15.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:14 smithi049 ceph-mon[32331]: from='client.? 172.21.15.115:0/3350434647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4828303-f823-4e3c-8298-33bba2eda98d"}]: dispatch 2024-03-26T17:58:15.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:14 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4828303-f823-4e3c-8298-33bba2eda98d"}]: dispatch 2024-03-26T17:58:15.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:14 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4828303-f823-4e3c-8298-33bba2eda98d"}]': finished 2024-03-26T17:58:15.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:14 smithi049 ceph-mon[32331]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T17:58:15.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:14 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:15.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:14 smithi171 ceph-mon[36033]: from='client.? 172.21.15.115:0/3350434647' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4828303-f823-4e3c-8298-33bba2eda98d"}]: dispatch 2024-03-26T17:58:15.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:14 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4828303-f823-4e3c-8298-33bba2eda98d"}]: dispatch 2024-03-26T17:58:15.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:14 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4828303-f823-4e3c-8298-33bba2eda98d"}]': finished 2024-03-26T17:58:15.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:14 smithi171 ceph-mon[36033]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T17:58:15.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:14 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:16.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:15 smithi115 ceph-mon[36174]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:16.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:15 smithi115 ceph-mon[36174]: from='client.? 172.21.15.115:0/2607418049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:58:16.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:15 smithi049 ceph-mon[32331]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:16.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:15 smithi049 ceph-mon[32331]: from='client.? 172.21.15.115:0/2607418049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:58:16.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:15 smithi171 ceph-mon[36033]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:16.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:15 smithi171 ceph-mon[36033]: from='client.? 172.21.15.115:0/2607418049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:58:18.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:17 smithi115 ceph-mon[36174]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:18.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:17 smithi049 ceph-mon[32331]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:18.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:17 smithi171 ceph-mon[36033]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:20.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:19 smithi115 ceph-mon[36174]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:20.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:19 smithi049 ceph-mon[32331]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:20.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:19 smithi171 ceph-mon[36033]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:22.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:21 smithi115 ceph-mon[36174]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:22.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:21 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T17:58:22.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:21 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:22.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:21 smithi049 ceph-mon[32331]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:22.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T17:58:22.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:22.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:21 smithi171 ceph-mon[36033]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:22.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:21 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T17:58:22.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:21 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:22.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:22 smithi115 ceph-mon[36174]: Deploying daemon osd.10 on smithi115 2024-03-26T17:58:23.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:22 smithi049 ceph-mon[32331]: Deploying daemon osd.10 on smithi115 2024-03-26T17:58:23.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:22 smithi171 ceph-mon[36033]: Deploying daemon osd.10 on smithi115 2024-03-26T17:58:23.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:23 smithi115 ceph-mon[36174]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:24.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:23 smithi049 ceph-mon[32331]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:24.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:23 smithi171 ceph-mon[36033]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:25.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:25 smithi115 ceph-mon[36174]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:26.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:25 smithi049 ceph-mon[32331]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:26.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:25 smithi171 ceph-mon[36033]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:28 smithi171 ceph-mon[36033]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:28.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:28 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:28.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:28 smithi115 ceph-mon[36174]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:28.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:28 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:28.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:28 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:28.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:28 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:28.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:28 smithi049 ceph-mon[32331]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:28.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:28 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:28.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:28 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:28.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:28 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:30.433 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:30 smithi115 ceph-mon[36174]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:30.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:30 smithi171 ceph-mon[36033]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:30.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:30 smithi049 ceph-mon[32331]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:30.830 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.845+0000 7effbdffb700 1 -- 172.21.15.115:0/13399834 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7effc00627f0 con 0x7effb00a3030 2024-03-26T17:58:30.830 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 10 on host 'smithi115' 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 -- 172.21.15.115:0/13399834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7effb00a3030 msgr2=0x7effb00a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7effb00a3030 0x7effb00a54f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7effac00b5c0 tx=0x7effac005fd0 comp rx=0 tx=0).stop 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 -- 172.21.15.115:0/13399834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effc0102ed0 msgr2=0x7effc0102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effc0102ed0 0x7effc0102830 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7effb400b530 tx=0x7effb400b840 comp rx=0 tx=0).stop 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 -- 172.21.15.115:0/13399834 shutdown_connections 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7effb00a3030 0x7effb00a54f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7effc0108130 0x7effc01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effc0102ed0 0x7effc0102830 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 --2- 172.21.15.115:0/13399834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0105800 0x7effc0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 -- 172.21.15.115:0/13399834 >> 172.21.15.115:0/13399834 conn(0x7effc00fc8a0 msgr2=0x7effc0103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 -- 172.21.15.115:0/13399834 shutdown_connections 2024-03-26T17:58:30.832 INFO:teuthology.orchestra.run.smithi115.stderr:2024-03-26T17:58:30.847+0000 7effc6fe0700 1 -- 172.21.15.115:0/13399834 wait complete. 2024-03-26T17:58:31.486 DEBUG:teuthology.orchestra.run.smithi115:osd.10> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.10.service 2024-03-26T17:58:31.488 INFO:tasks.cephadm:Deploying osd.11 on smithi171 with /dev/vg_nvme/lv_1... 2024-03-26T17:58:31.488 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T17:58:31.734 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:58:31.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:31 smithi115 ceph-mon[36174]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:31.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:31 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:31.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:31 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:31.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:31 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:31.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:31 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:31.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:31 smithi115 ceph-mon[36174]: from='osd.10 [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:58:31.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:31 smithi115 ceph-mon[36174]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:58:32.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:31 smithi171 ceph-mon[36033]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:32.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:31 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:32.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:31 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:32.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:31 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:32.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:31 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:32.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:31 smithi171 ceph-mon[36033]: from='osd.10 [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:58:32.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:31 smithi171 ceph-mon[36033]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:58:32.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:31 smithi049 ceph-mon[32331]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:32.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:31 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:32.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:31 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:32.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:31 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:32.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:31 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:32.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:31 smithi049 ceph-mon[32331]: from='osd.10 [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:58:32.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:31 smithi049 ceph-mon[32331]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:58:33.139 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:33 smithi171 ceph-mon[36033]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T17:58:33.140 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:33 smithi171 ceph-mon[36033]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T17:58:33.140 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:33.140 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:33 smithi171 ceph-mon[36033]: from='osd.10 [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:58:33.140 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:33 smithi171 ceph-mon[36033]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:58:33.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:33 smithi115 ceph-mon[36174]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T17:58:33.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:33 smithi115 ceph-mon[36174]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T17:58:33.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:33 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:33.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:33 smithi115 ceph-mon[36174]: from='osd.10 [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:58:33.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:33 smithi115 ceph-mon[36174]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:58:33.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:33 smithi049 ceph-mon[32331]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T17:58:33.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:33 smithi049 ceph-mon[32331]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T17:58:33.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:33.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:33 smithi049 ceph-mon[32331]: from='osd.10 [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:58:33.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:33 smithi049 ceph-mon[32331]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-03-26T17:58:33.843 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T17:58:33.843 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T17:58:33.844 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-03-26T17:58:33.844 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-03-26T17:58:33.844 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011315 s, 927 MB/s 2024-03-26T17:58:33.844 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-03-26T17:58:33.907 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 17:58:33 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T17:58:33.577+0000 7f3148e48700 -1 osd.10 0 waiting for initial osdmap 2024-03-26T17:58:33.907 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 17:58:33 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T17:58:33.589+0000 7f3140c33700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:58:34.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:34 smithi115 ceph-mon[36174]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:34.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:34 smithi115 ceph-mon[36174]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:58:34.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:34 smithi115 ceph-mon[36174]: osdmap e72: 11 total, 10 up, 11 in 2024-03-26T17:58:34.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:34 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:34.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:34 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:34.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:34 smithi115 ceph-mon[36174]: from='osd.10 ' entity='osd.10' 2024-03-26T17:58:34.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:34 smithi049 ceph-mon[32331]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:34.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:34 smithi049 ceph-mon[32331]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:58:34.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:34 smithi049 ceph-mon[32331]: osdmap e72: 11 total, 10 up, 11 in 2024-03-26T17:58:34.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:34 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:34.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:34 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:34.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:34 smithi049 ceph-mon[32331]: from='osd.10 ' entity='osd.10' 2024-03-26T17:58:34.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:34 smithi171 ceph-mon[36033]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:34.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:34 smithi171 ceph-mon[36033]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-03-26T17:58:34.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:34 smithi171 ceph-mon[36033]: osdmap e72: 11 total, 10 up, 11 in 2024-03-26T17:58:34.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:34 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:34.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:34 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:34.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:34 smithi171 ceph-mon[36033]: from='osd.10 ' entity='osd.10' 2024-03-26T17:58:34.640 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi171:vg_nvme/lv_1 2024-03-26T17:58:34.895 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:58:35.183 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: osd.10 [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] boot 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.184 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:35.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:35.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:35.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:35.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:35.185 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:58:35.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:58:35.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:35.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: osd.10 [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] boot 2024-03-26T17:58:35.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T17:58:35.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:35.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:35.490 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: osd.10 [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] boot 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:58:35.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:35.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:35.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:35.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:35.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:35.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:35.505 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:36.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:36 smithi049 ceph-mon[32331]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:36.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:36 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi115 2024-03-26T17:58:36.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:36 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi115 to 4270M 2024-03-26T17:58:36.405 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:36 smithi049 ceph-mon[32331]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T17:58:36.430 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:36 smithi115 ceph-mon[36174]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:36.431 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:36 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi115 2024-03-26T17:58:36.431 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:36 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi115 to 4270M 2024-03-26T17:58:36.431 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:36 smithi115 ceph-mon[36174]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T17:58:36.448 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:36 smithi171 ceph-mon[36033]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:58:36.448 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:36 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi115 2024-03-26T17:58:36.448 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:36 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi115 to 4270M 2024-03-26T17:58:36.448 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:36 smithi171 ceph-mon[36033]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T17:58:36.679 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.695+0000 7fb349e2d700 1 -- 172.21.15.171:0/831533618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb344102ef0 msgr2=0x7fb3441052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:36.680 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.695+0000 7fb349e2d700 1 --2- 172.21.15.171:0/831533618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb344102ef0 0x7fb3441052e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb33400b0a0 tx=0x7fb33400b3b0 comp rx=0 tx=0).stop 2024-03-26T17:58:36.680 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.695+0000 7fb349e2d700 1 -- 172.21.15.171:0/831533618 shutdown_connections 2024-03-26T17:58:36.680 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.695+0000 7fb349e2d700 1 --2- 172.21.15.171:0/831533618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb344108150 0x7fb34410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:36.680 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.695+0000 7fb349e2d700 1 --2- 172.21.15.171:0/831533618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb344102ef0 0x7fb3441052e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:36.680 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.695+0000 7fb349e2d700 1 --2- 172.21.15.171:0/831533618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344105820 0x7fb344107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:36.680 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.695+0000 7fb349e2d700 1 -- 172.21.15.171:0/831533618 >> 172.21.15.171:0/831533618 conn(0x7fb3440fc880 msgr2=0x7fb3440fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:36.680 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.695+0000 7fb349e2d700 1 -- 172.21.15.171:0/831533618 shutdown_connections 2024-03-26T17:58:36.680 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.695+0000 7fb349e2d700 1 -- 172.21.15.171:0/831533618 wait complete. 2024-03-26T17:58:36.680 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb349e2d700 1 Processor -- start 2024-03-26T17:58:36.680 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb349e2d700 1 -- start start 2024-03-26T17:58:36.682 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb349e2d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb344102ef0 0x7fb344198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:36.682 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb349e2d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344105820 0x7fb344198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb349e2d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb344108150 0x7fb34419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb349e2d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb34410e0b0 con 0x7fb344102ef0 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb349e2d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb34410df30 con 0x7fb344105820 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb349e2d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb34410e230 con 0x7fb344108150 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb343fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb344108150 0x7fb34419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb343fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb344108150 0x7fb34419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:47140/0 (socket says 172.21.15.171:47140) 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb343fff700 1 -- 172.21.15.171:0/1887432663 learned_addr learned my addr 172.21.15.171:0/1887432663 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb342ffd700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344105820 0x7fb344198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.696+0000 7fb3437fe700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb344102ef0 0x7fb344198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb343fff700 1 -- 172.21.15.171:0/1887432663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344105820 msgr2=0x7fb344198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:36.683 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb343fff700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344105820 0x7fb344198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:36.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb343fff700 1 -- 172.21.15.171:0/1887432663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb344102ef0 msgr2=0x7fb344198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:36.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb343fff700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb344102ef0 0x7fb344198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:36.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb343fff700 1 -- 172.21.15.171:0/1887432663 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3441a0360 con 0x7fb344108150 2024-03-26T17:58:36.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb343fff700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb344108150 0x7fb34419fcc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00b4b0 tx=0x7fb33c00b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:36.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb340ff9700 1 -- 172.21.15.171:0/1887432663 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c015020 con 0x7fb344108150 2024-03-26T17:58:36.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb340ff9700 1 -- 172.21.15.171:0/1887432663 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb33c011d10 con 0x7fb344108150 2024-03-26T17:58:36.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb349e2d700 1 -- 172.21.15.171:0/1887432663 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3441a0650 con 0x7fb344108150 2024-03-26T17:58:36.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb340ff9700 1 -- 172.21.15.171:0/1887432663 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c010d00 con 0x7fb344108150 2024-03-26T17:58:36.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.697+0000 7fb349e2d700 1 -- 172.21.15.171:0/1887432663 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3441a0c10 con 0x7fb344108150 2024-03-26T17:58:36.684 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.698+0000 7fb349e2d700 1 -- 172.21.15.171:0/1887432663 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb344069070 con 0x7fb344108150 2024-03-26T17:58:36.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.699+0000 7fb340ff9700 1 -- 172.21.15.171:0/1887432663 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fb33c018020 con 0x7fb344108150 2024-03-26T17:58:36.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.700+0000 7fb340ff9700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb32c0a2fb0 0x7fb32c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:36.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.700+0000 7fb340ff9700 1 -- 172.21.15.171:0/1887432663 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7fb33c0ce9f0 con 0x7fb344108150 2024-03-26T17:58:36.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.700+0000 7fb3437fe700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb32c0a2fb0 0x7fb32c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:36.686 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.701+0000 7fb3437fe700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb32c0a2fb0 0x7fb32c0a5470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb33400b070 tx=0x7fb33400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:36.687 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.703+0000 7fb340ff9700 1 -- 172.21.15.171:0/1887432663 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb33c01f020 con 0x7fb344108150 2024-03-26T17:58:36.890 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:36.906+0000 7fb349e2d700 1 -- 172.21.15.171:0/1887432663 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb3440627f0 con 0x7fb32c0a2fb0 2024-03-26T17:58:37.428 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:37 smithi049 ceph-mon[32331]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T17:58:37.428 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:37 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:58:37.428 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:37 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:58:37.428 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:37 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:37.443 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:37 smithi115 ceph-mon[36174]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T17:58:37.444 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:58:37.444 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:58:37.444 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:37 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:37.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:37 smithi171 ceph-mon[36033]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T17:58:37.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:37 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:58:37.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:37 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:58:37.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:37 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:38.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:38 smithi115 ceph-mon[36174]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:38.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:38 smithi115 ceph-mon[36174]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:58:38.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:38 smithi115 ceph-mon[36174]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T17:58:38.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:38 smithi171 ceph-mon[36033]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:38.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:38 smithi171 ceph-mon[36033]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:58:38.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:38 smithi171 ceph-mon[36033]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T17:58:38.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:38 smithi049 ceph-mon[32331]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:38.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:38 smithi049 ceph-mon[32331]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:58:38.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:38 smithi049 ceph-mon[32331]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T17:58:39.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:39 smithi115 ceph-mon[36174]: osdmap e77: 11 total, 11 up, 11 in 2024-03-26T17:58:39.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:39 smithi049 ceph-mon[32331]: osdmap e77: 11 total, 11 up, 11 in 2024-03-26T17:58:39.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:39 smithi171 ceph-mon[36033]: osdmap e77: 11 total, 11 up, 11 in 2024-03-26T17:58:40.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:40 smithi171 ceph-mon[36033]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T17:58:40.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:40 smithi171 ceph-mon[36033]: from='client.? 172.21.15.171:0/1865070635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f67590c-f8b2-4ffb-84e5-825d76a17804"}]: dispatch 2024-03-26T17:58:40.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:40 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f67590c-f8b2-4ffb-84e5-825d76a17804"}]: dispatch 2024-03-26T17:58:40.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:40 smithi171 ceph-mon[36033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f67590c-f8b2-4ffb-84e5-825d76a17804"}]': finished 2024-03-26T17:58:40.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:40 smithi171 ceph-mon[36033]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T17:58:40.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:40 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:58:40.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:40 smithi115 ceph-mon[36174]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T17:58:40.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:40 smithi115 ceph-mon[36174]: from='client.? 172.21.15.171:0/1865070635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f67590c-f8b2-4ffb-84e5-825d76a17804"}]: dispatch 2024-03-26T17:58:40.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:40 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f67590c-f8b2-4ffb-84e5-825d76a17804"}]: dispatch 2024-03-26T17:58:40.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:40 smithi115 ceph-mon[36174]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f67590c-f8b2-4ffb-84e5-825d76a17804"}]': finished 2024-03-26T17:58:40.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:40 smithi115 ceph-mon[36174]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T17:58:40.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:40 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:58:40.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:40 smithi049 ceph-mon[32331]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T17:58:40.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:40 smithi049 ceph-mon[32331]: from='client.? 172.21.15.171:0/1865070635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f67590c-f8b2-4ffb-84e5-825d76a17804"}]: dispatch 2024-03-26T17:58:40.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:40 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f67590c-f8b2-4ffb-84e5-825d76a17804"}]: dispatch 2024-03-26T17:58:40.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:40 smithi049 ceph-mon[32331]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f67590c-f8b2-4ffb-84e5-825d76a17804"}]': finished 2024-03-26T17:58:40.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:40 smithi049 ceph-mon[32331]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T17:58:40.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:40 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:58:41.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:41 smithi115 ceph-mon[36174]: from='client.? 172.21.15.171:0/3970911128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:58:41.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:41 smithi049 ceph-mon[32331]: from='client.? 172.21.15.171:0/3970911128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:58:41.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:41 smithi171 ceph-mon[36033]: from='client.? 172.21.15.171:0/3970911128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:58:42.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:42 smithi115 ceph-mon[36174]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-26T17:58:42.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:42 smithi049 ceph-mon[32331]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-26T17:58:42.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:42 smithi171 ceph-mon[36033]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-26T17:58:44.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:44 smithi171 ceph-mon[36033]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:58:44.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:44 smithi115 ceph-mon[36174]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:58:44.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:44 smithi049 ceph-mon[32331]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:58:46.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:46 smithi171 ceph-mon[36033]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-26T17:58:46.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:46 smithi115 ceph-mon[36174]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-26T17:58:46.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:46 smithi049 ceph-mon[32331]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-26T17:58:48.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:48 smithi171 ceph-mon[36033]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:48.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:48 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T17:58:48.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:48 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:48.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:48 smithi171 ceph-mon[36033]: Deploying daemon osd.11 on smithi171 2024-03-26T17:58:48.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:48 smithi115 ceph-mon[36174]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:48.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:48 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T17:58:48.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:48 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:48.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:48 smithi115 ceph-mon[36174]: Deploying daemon osd.11 on smithi171 2024-03-26T17:58:48.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:48 smithi049 ceph-mon[32331]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:48.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:48 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T17:58:48.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:48 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:48.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:48 smithi049 ceph-mon[32331]: Deploying daemon osd.11 on smithi171 2024-03-26T17:58:50.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:50 smithi115 ceph-mon[36174]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:50.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:50 smithi049 ceph-mon[32331]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:50.526 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:50 smithi171 ceph-mon[36033]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:52.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:52 smithi115 ceph-mon[36174]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:52.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:52 smithi049 ceph-mon[32331]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:52.572 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:52 smithi171 ceph-mon[36033]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:54.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:54 smithi171 ceph-mon[36033]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:54.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:54.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:54.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:54 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:54.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:54 smithi115 ceph-mon[36174]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:54.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:54.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:54.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:54 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:54.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:54 smithi049 ceph-mon[32331]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:54.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:54.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:54.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:54 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:56.297 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:56 smithi171 ceph-mon[36033]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:56.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:56 smithi115 ceph-mon[36174]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:56.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:56 smithi049 ceph-mon[32331]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:56.661 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.676+0000 7fb340ff9700 1 -- 172.21.15.171:0/1887432663 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb3440627f0 con 0x7fb32c0a2fb0 2024-03-26T17:58:56.661 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 11 on host 'smithi171' 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 -- 172.21.15.171:0/1887432663 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb32c0a2fb0 msgr2=0x7fb32c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb32c0a2fb0 0x7fb32c0a5470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb33400b070 tx=0x7fb33400b7b0 comp rx=0 tx=0).stop 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 -- 172.21.15.171:0/1887432663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb344108150 msgr2=0x7fb34419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb344108150 0x7fb34419fcc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00b4b0 tx=0x7fb33c00b7c0 comp rx=0 tx=0).stop 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 -- 172.21.15.171:0/1887432663 shutdown_connections 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb32c0a2fb0 0x7fb32c0a5470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb344108150 0x7fb34419fcc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb344102ef0 0x7fb344198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 --2- 172.21.15.171:0/1887432663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb344105820 0x7fb344198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 -- 172.21.15.171:0/1887432663 >> 172.21.15.171:0/1887432663 conn(0x7fb3440fc880 msgr2=0x7fb344103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 -- 172.21.15.171:0/1887432663 shutdown_connections 2024-03-26T17:58:56.664 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:58:56.679+0000 7fb349e2d700 1 -- 172.21.15.171:0/1887432663 wait complete. 2024-03-26T17:58:57.303 DEBUG:teuthology.orchestra.run.smithi171:osd.11> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.11.service 2024-03-26T17:58:57.305 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-26T17:58:57.305 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T17:58:57.570 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:58:57.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:57.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:57.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:57 smithi049 ceph-mon[32331]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:57.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:57 smithi049 ceph-mon[32331]: from='osd.11 [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:58:57.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:57 smithi049 ceph-mon[32331]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:58:57.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:57.952 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:57 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:57.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:57.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:57.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:57 smithi115 ceph-mon[36174]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:57.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:57 smithi115 ceph-mon[36174]: from='osd.11 [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:58:57.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:57 smithi115 ceph-mon[36174]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:58:57.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:57.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:57 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:58.095 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:58.095 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:58.095 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:57 smithi171 ceph-mon[36033]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:58.095 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:57 smithi171 ceph-mon[36033]: from='osd.11 [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:58:58.095 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:57 smithi171 ceph-mon[36033]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:58:58.095 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:58.095 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:57 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:58:58.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:58 smithi171 ceph-mon[36033]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T17:58:58.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:58 smithi171 ceph-mon[36033]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T17:58:58.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:58 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:58:58.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:58 smithi171 ceph-mon[36033]: from='osd.11 [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:58:58.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:58 smithi171 ceph-mon[36033]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:58:58.858 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 17:58:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T17:58:58.645+0000 7fc5447f5700 -1 osd.11 0 waiting for initial osdmap 2024-03-26T17:58:58.858 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 17:58:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T17:58:58.659+0000 7fc5405e8700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:58:58.970 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:58 smithi049 ceph-mon[32331]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T17:58:58.970 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:58 smithi049 ceph-mon[32331]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T17:58:58.970 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:58 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:58:58.970 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:58 smithi049 ceph-mon[32331]: from='osd.11 [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:58:58.970 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:58 smithi049 ceph-mon[32331]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:58:58.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:58 smithi115 ceph-mon[36174]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T17:58:58.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:58 smithi115 ceph-mon[36174]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T17:58:58.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:58 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:58:58.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:58 smithi115 ceph-mon[36174]: from='osd.11 [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:58:58.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:58 smithi115 ceph-mon[36174]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-03-26T17:58:59.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.479+0000 7efdc2886700 1 -- 172.21.15.49:0/1995087061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc102ea0 msgr2=0x7efdbc105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:59.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.479+0000 7efdc2886700 1 --2- 172.21.15.49:0/1995087061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc102ea0 0x7efdbc105290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efdac009990 tx=0x7efdac009ca0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.479+0000 7efdc2886700 1 -- 172.21.15.49:0/1995087061 shutdown_connections 2024-03-26T17:58:59.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.479+0000 7efdc2886700 1 --2- 172.21.15.49:0/1995087061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdbc108100 0x7efdbc10a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.479+0000 7efdc2886700 1 --2- 172.21.15.49:0/1995087061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc102ea0 0x7efdbc105290 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.479+0000 7efdc2886700 1 --2- 172.21.15.49:0/1995087061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc1057d0 0x7efdbc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.479+0000 7efdc2886700 1 -- 172.21.15.49:0/1995087061 >> 172.21.15.49:0/1995087061 conn(0x7efdbc0fc8d0 msgr2=0x7efdbc0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:59.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.479+0000 7efdc2886700 1 -- 172.21.15.49:0/1995087061 shutdown_connections 2024-03-26T17:58:59.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.479+0000 7efdc2886700 1 -- 172.21.15.49:0/1995087061 wait complete. 2024-03-26T17:58:59.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.479+0000 7efdc2886700 1 Processor -- start 2024-03-26T17:58:59.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdc2886700 1 -- start start 2024-03-26T17:58:59.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdc2886700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdbc102ea0 0x7efdbc102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:59.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdc2886700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1057d0 0x7efdbc100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdc2886700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc108100 0x7efdbc101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdc2886700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc10e020 con 0x7efdbc1057d0 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdc2886700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc10dea0 con 0x7efdbc108100 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdc2886700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc10e1a0 con 0x7efdbc102ea0 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdbc102ea0 0x7efdbc102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdbc102ea0 0x7efdbc102800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:47802/0 (socket says 172.21.15.49:47802) 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdbb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1057d0 0x7efdbc100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdbbfff700 1 -- 172.21.15.49:0/2100225463 learned_addr learned my addr 172.21.15.49:0/2100225463 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdc0e23700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc108100 0x7efdbc101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdbbfff700 1 -- 172.21.15.49:0/2100225463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc108100 msgr2=0x7efdbc101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdbbfff700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc108100 0x7efdbc101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdbbfff700 1 -- 172.21.15.49:0/2100225463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1057d0 msgr2=0x7efdbc100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdbbfff700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1057d0 0x7efdbc100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdbbfff700 1 -- 172.21.15.49:0/2100225463 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc101c40 con 0x7efdbc102ea0 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdbb7fe700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1057d0 0x7efdbc100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.480+0000 7efdc0e23700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc108100 0x7efdbc101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.481+0000 7efdbbfff700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdbc102ea0 0x7efdbc102800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efdac005910 tx=0x7efdac009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.481+0000 7efdb97fa700 1 -- 172.21.15.49:0/2100225463 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac027070 con 0x7efdbc102ea0 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.481+0000 7efdb97fa700 1 -- 172.21.15.49:0/2100225463 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdac004150 con 0x7efdbc102ea0 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.481+0000 7efdc2886700 1 -- 172.21.15.49:0/2100225463 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdbc1a2720 con 0x7efdbc102ea0 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.481+0000 7efdb97fa700 1 -- 172.21.15.49:0/2100225463 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac00f460 con 0x7efdbc102ea0 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.481+0000 7efdc2886700 1 -- 172.21.15.49:0/2100225463 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdbc1a2b70 con 0x7efdbc102ea0 2024-03-26T17:58:59.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.482+0000 7efdc2886700 1 -- 172.21.15.49:0/2100225463 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdbc069070 con 0x7efdbc102ea0 2024-03-26T17:58:59.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.483+0000 7efdb97fa700 1 -- 172.21.15.49:0/2100225463 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7efdac02a070 con 0x7efdbc102ea0 2024-03-26T17:58:59.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.484+0000 7efdb97fa700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efda80a2fb0 0x7efda80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:59.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.484+0000 7efdbb7fe700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efda80a2fb0 0x7efda80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:59.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.484+0000 7efdb97fa700 1 -- 172.21.15.49:0/2100225463 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7efdac0d0510 con 0x7efdbc102ea0 2024-03-26T17:58:59.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.484+0000 7efdbb7fe700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efda80a2fb0 0x7efda80a5470 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efda4010420 tx=0x7efda40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:59.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.487+0000 7efdb97fa700 1 -- 172.21.15.49:0/2100225463 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efdac031070 con 0x7efdbc102ea0 2024-03-26T17:58:59.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.694+0000 7efdc2886700 1 -- 172.21.15.49:0/2100225463 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7efdbc062fc0 con 0x7efdbc102ea0 2024-03-26T17:58:59.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.695+0000 7efdb97fa700 1 -- 172.21.15.49:0/2100225463 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7efdac013070 con 0x7efdbc102ea0 2024-03-26T17:58:59.679 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:58:59.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 -- 172.21.15.49:0/2100225463 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efda80a2fb0 msgr2=0x7efda80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:59.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efda80a2fb0 0x7efda80a5470 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efda4010420 tx=0x7efda40073d0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 -- 172.21.15.49:0/2100225463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdbc102ea0 msgr2=0x7efdbc102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:59.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdbc102ea0 0x7efdbc102800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efdac005910 tx=0x7efdac009580 comp rx=0 tx=0).stop 2024-03-26T17:58:59.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 -- 172.21.15.49:0/2100225463 shutdown_connections 2024-03-26T17:58:59.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efda80a2fb0 0x7efda80a5470 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efdbc108100 0x7efdbc101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdbc102ea0 0x7efdbc102800 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 --2- 172.21.15.49:0/2100225463 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efdbc1057d0 0x7efdbc100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:59.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 -- 172.21.15.49:0/2100225463 >> 172.21.15.49:0/2100225463 conn(0x7efdbc0fc8d0 msgr2=0x7efdbc103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:59.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 -- 172.21.15.49:0/2100225463 shutdown_connections 2024-03-26T17:58:59.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:58:59.697+0000 7efdc2886700 1 -- 172.21.15.49:0/2100225463 wait complete. 2024-03-26T17:58:59.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:59 smithi115 ceph-mon[36174]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:58:59.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:59 smithi115 ceph-mon[36174]: osdmap e80: 12 total, 11 up, 12 in 2024-03-26T17:58:59.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:59 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:58:59.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:59 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:58:59.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:58:59 smithi115 ceph-mon[36174]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:58:59.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:59 smithi049 ceph-mon[32331]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:58:59.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:59 smithi049 ceph-mon[32331]: osdmap e80: 12 total, 11 up, 12 in 2024-03-26T17:58:59.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:58:59.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:59 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:58:59.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:58:59 smithi049 ceph-mon[32331]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:59:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:59 smithi171 ceph-mon[36033]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-03-26T17:59:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:59 smithi171 ceph-mon[36033]: osdmap e80: 12 total, 11 up, 12 in 2024-03-26T17:59:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:59 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:59:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:59 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:59:00.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:58:59 smithi171 ceph-mon[36033]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:59:00.436 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1711475939,"num_in_osds":12,"osd_in_since":1711475919,"num_remapped_pgs":0} 2024-03-26T17:59:00.437 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T17:59:00.700 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:59:00.739 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:00 smithi049 ceph-mon[32331]: purged_snaps scrub starts 2024-03-26T17:59:00.739 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:00 smithi049 ceph-mon[32331]: purged_snaps scrub ok 2024-03-26T17:59:00.739 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:00 smithi049 ceph-mon[32331]: osd.11 [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] boot 2024-03-26T17:59:00.739 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:00 smithi049 ceph-mon[32331]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T17:59:00.739 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:00 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:59:00.739 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:00 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2100225463' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T17:59:00.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:00 smithi115 ceph-mon[36174]: purged_snaps scrub starts 2024-03-26T17:59:00.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:00 smithi115 ceph-mon[36174]: purged_snaps scrub ok 2024-03-26T17:59:00.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:00 smithi115 ceph-mon[36174]: osd.11 [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] boot 2024-03-26T17:59:00.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:00 smithi115 ceph-mon[36174]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T17:59:00.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:00 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:59:00.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:00 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2100225463' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T17:59:00.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:00 smithi171 ceph-mon[36033]: purged_snaps scrub starts 2024-03-26T17:59:00.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:00 smithi171 ceph-mon[36033]: purged_snaps scrub ok 2024-03-26T17:59:00.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:00 smithi171 ceph-mon[36033]: osd.11 [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] boot 2024-03-26T17:59:00.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:00 smithi171 ceph-mon[36033]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T17:59:00.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:00 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:59:00.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:00 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2100225463' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T17:59:01.855 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T17:59:01.855 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:01.855 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: Detected new or changed devices on smithi171 2024-03-26T17:59:01.855 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:01.855 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:01.855 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi171 to 4270M 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:01.856 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:01 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:01.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T17:59:01.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:01.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: Detected new or changed devices on smithi171 2024-03-26T17:59:01.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi171 to 4270M 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:01.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:01 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:02.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T17:59:02.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: Detected new or changed devices on smithi171 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi171 to 4270M 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:02.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:02.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:02.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:01 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.404+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/722940786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44103040 msgr2=0x7f2e44107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.404+0000 7f2e4a0d2700 1 --2- 172.21.15.49:0/722940786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44103040 0x7f2e44107010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2e380099c0 tx=0x7f2e38009cd0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.404+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/722940786 shutdown_connections 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.404+0000 7f2e4a0d2700 1 --2- 172.21.15.49:0/722940786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e44107740 0x7f2e4410f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.404+0000 7f2e4a0d2700 1 --2- 172.21.15.49:0/722940786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e44102690 0x7f2e44102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.404+0000 7f2e4a0d2700 1 --2- 172.21.15.49:0/722940786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44103040 0x7f2e44107010 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.404+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/722940786 >> 172.21.15.49:0/722940786 conn(0x7f2e440794d0 msgr2=0x7f2e440798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.404+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/722940786 shutdown_connections 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.404+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/722940786 wait complete. 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.405+0000 7f2e4a0d2700 1 Processor -- start 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.405+0000 7f2e4a0d2700 1 -- start start 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.405+0000 7f2e4a0d2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e44102690 0x7f2e4419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.405+0000 7f2e4a0d2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e44103040 0x7f2e4419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.405+0000 7f2e4a0d2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44107740 0x7f2e44198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.405+0000 7f2e4a0d2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441126b0 con 0x7f2e44107740 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.405+0000 7f2e4a0d2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44112530 con 0x7f2e44103040 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.405+0000 7f2e4a0d2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44112830 con 0x7f2e44102690 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e43fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44107740 0x7f2e44198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e43fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44107740 0x7f2e44198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60818/0 (socket says 172.21.15.49:60818) 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e43fff700 1 -- 172.21.15.49:0/1574132721 learned_addr learned my addr 172.21.15.49:0/1574132721 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e437fe700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e44102690 0x7f2e4419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e42ffd700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e44103040 0x7f2e4419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e43fff700 1 -- 172.21.15.49:0/1574132721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e44102690 msgr2=0x7f2e4419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e43fff700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e44102690 0x7f2e4419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e43fff700 1 -- 172.21.15.49:0/1574132721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e44103040 msgr2=0x7f2e4419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:02.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e43fff700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e44103040 0x7f2e4419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e43fff700 1 -- 172.21.15.49:0/1574132721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e441990f0 con 0x7f2e44107740 2024-03-26T17:59:02.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e42ffd700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e44103040 0x7f2e4419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:02.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e43fff700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44107740 0x7f2e44198990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c007ee0 tx=0x7f2e3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:02.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e40ff9700 1 -- 172.21.15.49:0/1574132721 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00dbd0 con 0x7f2e44107740 2024-03-26T17:59:02.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e40ff9700 1 -- 172.21.15.49:0/1574132721 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e3c00dd70 con 0x7f2e44107740 2024-03-26T17:59:02.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/1574132721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441993e0 con 0x7f2e44107740 2024-03-26T17:59:02.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e40ff9700 1 -- 172.21.15.49:0/1574132721 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c005a60 con 0x7f2e44107740 2024-03-26T17:59:02.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.406+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/1574132721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a56a0 con 0x7f2e44107740 2024-03-26T17:59:02.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.407+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/1574132721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4410a8f0 con 0x7f2e44107740 2024-03-26T17:59:02.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.408+0000 7f2e40ff9700 1 -- 172.21.15.49:0/1574132721 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f2e3c01a020 con 0x7f2e44107740 2024-03-26T17:59:02.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.409+0000 7f2e40ff9700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2e2c0a2fb0 0x7f2e2c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:02.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.409+0000 7f2e437fe700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2e2c0a2fb0 0x7f2e2c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:02.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.409+0000 7f2e40ff9700 1 -- 172.21.15.49:0/1574132721 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2e3c0ce640 con 0x7f2e44107740 2024-03-26T17:59:02.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.409+0000 7f2e437fe700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2e2c0a2fb0 0x7f2e2c0a5470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e44199ad0 tx=0x7f2e3400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:02.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.414+0000 7f2e40ff9700 1 -- 172.21.15.49:0/1574132721 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2e3c010070 con 0x7f2e44107740 2024-03-26T17:59:02.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.619+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/1574132721 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e4410a2d0 con 0x7f2e44107740 2024-03-26T17:59:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.620+0000 7f2e40ff9700 1 -- 172.21.15.49:0/1574132721 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18464 (secure 0 0 0) 0x7f2e3c09b910 con 0x7f2e44107740 2024-03-26T17:59:02.605 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:02.605 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:00.651658+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.622+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/1574132721 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2e2c0a2fb0 msgr2=0x7f2e2c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.622+0000 7f2e4a0d2700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2e2c0a2fb0 0x7f2e2c0a5470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e44199ad0 tx=0x7f2e3400b3c0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.622+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/1574132721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44107740 msgr2=0x7f2e44198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.622+0000 7f2e4a0d2700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44107740 0x7f2e44198990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c007ee0 tx=0x7f2e3c00d3b0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.622+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/1574132721 shutdown_connections 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.622+0000 7f2e4a0d2700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2e2c0a2fb0 0x7f2e2c0a5470 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.622+0000 7f2e4a0d2700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e44107740 0x7f2e44198990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.622+0000 7f2e4a0d2700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e44102690 0x7f2e4419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.622+0000 7f2e4a0d2700 1 --2- 172.21.15.49:0/1574132721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e44103040 0x7f2e4419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.622+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/1574132721 >> 172.21.15.49:0/1574132721 conn(0x7f2e440794d0 msgr2=0x7f2e44108870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:02.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.623+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/1574132721 shutdown_connections 2024-03-26T17:59:02.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:02.623+0000 7f2e4a0d2700 1 -- 172.21.15.49:0/1574132721 wait complete. 2024-03-26T17:59:02.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:02 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1574132721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:03.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:02 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1574132721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:03.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:02 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1574132721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:03.270 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:59:03.271 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-26T17:59:03.523 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:59:03.908 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:03 smithi049 ceph-mon[32331]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:03.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:03 smithi115 ceph-mon[36174]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:04.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:03 smithi171 ceph-mon[36033]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:05.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.249+0000 7fa14615d700 1 -- 172.21.15.49:0/3934526498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14006aa00 msgr2=0x7fa14010f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:05.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.249+0000 7fa14615d700 1 --2- 172.21.15.49:0/3934526498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14006aa00 0x7fa14010f240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b0d0 tx=0x7fa13000b3e0 comp rx=0 tx=0).stop 2024-03-26T17:59:05.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.249+0000 7fa14615d700 1 -- 172.21.15.49:0/3934526498 shutdown_connections 2024-03-26T17:59:05.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.249+0000 7fa14615d700 1 --2- 172.21.15.49:0/3934526498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14006aa00 0x7fa14010f240 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:05.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.249+0000 7fa14615d700 1 --2- 172.21.15.49:0/3934526498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa140108400 0x7fa1401087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:05.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.249+0000 7fa14615d700 1 --2- 172.21.15.49:0/3934526498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa14006a060 0x7fa14006a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:05.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.249+0000 7fa14615d700 1 -- 172.21.15.49:0/3934526498 >> 172.21.15.49:0/3934526498 conn(0x7fa1400781e0 msgr2=0x7fa1400785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:05.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.249+0000 7fa14615d700 1 -- 172.21.15.49:0/3934526498 shutdown_connections 2024-03-26T17:59:05.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.249+0000 7fa14615d700 1 -- 172.21.15.49:0/3934526498 wait complete. 2024-03-26T17:59:05.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.250+0000 7fa14615d700 1 Processor -- start 2024-03-26T17:59:05.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.250+0000 7fa14615d700 1 -- start start 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.250+0000 7fa14615d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14006a060 0x7fa14019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.250+0000 7fa14615d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa14006aa00 0x7fa14019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.250+0000 7fa14615d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa140108400 0x7fa140198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.250+0000 7fa14615d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa140112680 con 0x7fa14006a060 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.250+0000 7fa14615d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa140112500 con 0x7fa140108400 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.250+0000 7fa14615d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa140112800 con 0x7fa14006aa00 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14006a060 0x7fa14019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14006a060 0x7fa14019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60838/0 (socket says 172.21.15.49:60838) 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13f7fe700 1 -- 172.21.15.49:0/1920982562 learned_addr learned my addr 172.21.15.49:0/1920982562 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13effd700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa14006aa00 0x7fa14019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13ffff700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa140108400 0x7fa140198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:05.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13f7fe700 1 -- 172.21.15.49:0/1920982562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa14006aa00 msgr2=0x7fa14019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13f7fe700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa14006aa00 0x7fa14019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13f7fe700 1 -- 172.21.15.49:0/1920982562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa140108400 msgr2=0x7fa140198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13f7fe700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa140108400 0x7fa140198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13f7fe700 1 -- 172.21.15.49:0/1920982562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa140199080 con 0x7fa14006a060 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13f7fe700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14006a060 0x7fa14019e760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa13800d980 tx=0x7fa13800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13cff9700 1 -- 172.21.15.49:0/1920982562 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138017070 con 0x7fa14006a060 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa14615d700 1 -- 172.21.15.49:0/1920982562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa140199370 con 0x7fa14006a060 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.251+0000 7fa13cff9700 1 -- 172.21.15.49:0/1920982562 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa138005020 con 0x7fa14006a060 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.252+0000 7fa14615d700 1 -- 172.21.15.49:0/1920982562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1401a5630 con 0x7fa14006a060 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.252+0000 7fa13cff9700 1 -- 172.21.15.49:0/1920982562 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13800fbf0 con 0x7fa14006a060 2024-03-26T17:59:05.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.253+0000 7fa14615d700 1 -- 172.21.15.49:0/1920982562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa120005330 con 0x7fa14006a060 2024-03-26T17:59:05.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.254+0000 7fa13cff9700 1 -- 172.21.15.49:0/1920982562 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fa138004a30 con 0x7fa14006a060 2024-03-26T17:59:05.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.254+0000 7fa13cff9700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa12c0a3030 0x7fa12c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:05.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.254+0000 7fa13effd700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa12c0a3030 0x7fa12c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:05.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.254+0000 7fa13cff9700 1 -- 172.21.15.49:0/1920982562 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1380ceba0 con 0x7fa14006a060 2024-03-26T17:59:05.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.255+0000 7fa13effd700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa12c0a3030 0x7fa12c0a54f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa140078770 tx=0x7fa12800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:05.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.259+0000 7fa13cff9700 1 -- 172.21.15.49:0/1920982562 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa138020020 con 0x7fa14006a060 2024-03-26T17:59:05.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.464+0000 7fa14615d700 1 -- 172.21.15.49:0/1920982562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1200058f0 con 0x7fa14006a060 2024-03-26T17:59:05.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.465+0000 7fa13cff9700 1 -- 172.21.15.49:0/1920982562 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fa13809be70 con 0x7fa14006a060 2024-03-26T17:59:05.449 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T17:59:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 -- 172.21.15.49:0/1920982562 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa12c0a3030 msgr2=0x7fa12c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa12c0a3030 0x7fa12c0a54f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa140078770 tx=0x7fa12800b410 comp rx=0 tx=0).stop 2024-03-26T17:59:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 -- 172.21.15.49:0/1920982562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14006a060 msgr2=0x7fa14019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:05.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14006a060 0x7fa14019e760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa13800d980 tx=0x7fa13800dc90 comp rx=0 tx=0).stop 2024-03-26T17:59:05.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 -- 172.21.15.49:0/1920982562 shutdown_connections 2024-03-26T17:59:05.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa12c0a3030 0x7fa12c0a54f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:05.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa140108400 0x7fa140198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:05.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa14006a060 0x7fa14019e760 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:05.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 --2- 172.21.15.49:0/1920982562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa14006aa00 0x7fa14019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:05.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 -- 172.21.15.49:0/1920982562 >> 172.21.15.49:0/1920982562 conn(0x7fa1400781e0 msgr2=0x7fa1400ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:05.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 -- 172.21.15.49:0/1920982562 shutdown_connections 2024-03-26T17:59:05.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:05.467+0000 7fa14615d700 1 -- 172.21.15.49:0/1920982562 wait complete. 2024-03-26T17:59:05.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:05 smithi115 ceph-mon[36174]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:05.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:05 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1920982562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:59:06.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:05 smithi049 ceph-mon[32331]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:06.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:05 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1920982562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:59:06.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:05 smithi171 ceph-mon[36033]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:06.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:05 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1920982562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:59:06.213 INFO:tasks.cephadm:Adding mds.a on smithi049 2024-03-26T17:59:06.213 INFO:tasks.cephadm:Adding mds.d on smithi049 2024-03-26T17:59:06.214 INFO:tasks.cephadm:Adding mds.g on smithi049 2024-03-26T17:59:06.214 INFO:tasks.cephadm:Adding mds.j on smithi049 2024-03-26T17:59:06.214 INFO:tasks.cephadm:Adding mds.b on smithi115 2024-03-26T17:59:06.214 INFO:tasks.cephadm:Adding mds.e on smithi115 2024-03-26T17:59:06.214 INFO:tasks.cephadm:Adding mds.h on smithi115 2024-03-26T17:59:06.214 INFO:tasks.cephadm:Adding mds.k on smithi115 2024-03-26T17:59:06.214 INFO:tasks.cephadm:Adding mds.c on smithi171 2024-03-26T17:59:06.214 INFO:tasks.cephadm:Adding mds.f on smithi171 2024-03-26T17:59:06.214 INFO:tasks.cephadm:Adding mds.i on smithi171 2024-03-26T17:59:06.214 INFO:tasks.cephadm:Adding mds.l on smithi171 2024-03-26T17:59:06.214 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l' 2024-03-26T17:59:06.467 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:59:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:07 smithi171 ceph-mon[36033]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:07.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:07 smithi115 ceph-mon[36174]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:08.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:07 smithi049 ceph-mon[32331]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:08.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 -- 172.21.15.171:0/1955394831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c8102eb0 msgr2=0x7f94c8109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:08.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 --2- 172.21.15.171:0/1955394831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c8102eb0 0x7f94c8109760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94c000b0a0 tx=0x7f94c000b3b0 comp rx=0 tx=0).stop 2024-03-26T17:59:08.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 -- 172.21.15.171:0/1955394831 shutdown_connections 2024-03-26T17:59:08.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 --2- 172.21.15.171:0/1955394831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c8102eb0 0x7f94c8109760 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:08.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 --2- 172.21.15.171:0/1955394831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c8101310 0x7f94c8101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:08.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 --2- 172.21.15.171:0/1955394831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8102510 0x7f94c8102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:08.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 -- 172.21.15.171:0/1955394831 >> 172.21.15.171:0/1955394831 conn(0x7f94c80fc8d0 msgr2=0x7f94c80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:08.115 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 -- 172.21.15.171:0/1955394831 shutdown_connections 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 -- 172.21.15.171:0/1955394831 wait complete. 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 Processor -- start 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.129+0000 7f94ce472700 1 -- start start 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94ce472700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8101310 0x7f94c8105310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94ce472700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c8102510 0x7f94c8103960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94ce472700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c8102eb0 0x7f94c8103ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94ce472700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f94c810cee0 con 0x7f94c8102eb0 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94ce472700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f94c810cd60 con 0x7f94c8101310 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94ce472700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94c810d060 con 0x7f94c8102510 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94c77fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c8102510 0x7f94c8103960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94c77fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c8102510 0x7f94c8103960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:43424/0 (socket says 172.21.15.171:43424) 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94c77fe700 1 -- 172.21.15.171:0/1637726043 learned_addr learned my addr 172.21.15.171:0/1637726043 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94c7fff700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8101310 0x7f94c8105310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:08.116 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94cca0f700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c8102eb0 0x7f94c8103ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94c77fe700 1 -- 172.21.15.171:0/1637726043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8101310 msgr2=0x7f94c8105310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94c77fe700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8101310 0x7f94c8105310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94c77fe700 1 -- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c8102eb0 msgr2=0x7f94c8103ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94c77fe700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c8102eb0 0x7f94c8103ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94c77fe700 1 -- 172.21.15.171:0/1637726043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c8104750 con 0x7f94c8102510 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94cca0f700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c8102eb0 0x7f94c8103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.130+0000 7f94c77fe700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c8102510 0x7f94c8103960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d950 tx=0x7f94bc00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.131+0000 7f94c57fa700 1 -- 172.21.15.171:0/1637726043 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc016070 con 0x7f94c8102510 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.131+0000 7f94ce472700 1 -- 172.21.15.171:0/1637726043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c81a26e0 con 0x7f94c8102510 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.131+0000 7f94c57fa700 1 -- 172.21.15.171:0/1637726043 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94bc009e20 con 0x7f94c8102510 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.131+0000 7f94c57fa700 1 -- 172.21.15.171:0/1637726043 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc00f620 con 0x7f94c8102510 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.131+0000 7f94ce472700 1 -- 172.21.15.171:0/1637726043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c81a2c20 con 0x7f94c8102510 2024-03-26T17:59:08.117 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.131+0000 7f94ce472700 1 -- 172.21.15.171:0/1637726043 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c8069070 con 0x7f94c8102510 2024-03-26T17:59:08.118 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.133+0000 7f94c57fa700 1 -- 172.21.15.171:0/1637726043 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f94bc019070 con 0x7f94c8102510 2024-03-26T17:59:08.118 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.133+0000 7f94c57fa700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f94b00a2fb0 0x7f94b00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:08.118 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.133+0000 7f94c57fa700 1 -- 172.21.15.171:0/1637726043 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f94bc0cefe0 con 0x7f94c8102510 2024-03-26T17:59:08.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.133+0000 7f94c7fff700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f94b00a2fb0 0x7f94b00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:08.119 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.134+0000 7f94c7fff700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f94b00a2fb0 0x7f94b00a5470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94c8102370 tx=0x7f94b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:08.120 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.137+0000 7f94c57fa700 1 -- 172.21.15.171:0/1637726043 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f94bc09c2b0 con 0x7f94c8102510 2024-03-26T17:59:08.326 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.342+0000 7f94ce472700 1 -- 172.21.15.171:0/1637726043 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}) -- 0x7f94c806b2a0 con 0x7f94b00a2fb0 2024-03-26T17:59:08.334 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.350+0000 7f94c57fa700 1 -- 172.21.15.171:0/1637726043 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f94c806b2a0 con 0x7f94b00a2fb0 2024-03-26T17:59:08.334 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mds.all update... 2024-03-26T17:59:08.337 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 -- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f94b00a2fb0 msgr2=0x7f94b00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:08.337 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f94b00a2fb0 0x7f94b00a5470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94c8102370 tx=0x7f94b800b410 comp rx=0 tx=0).stop 2024-03-26T17:59:08.337 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 -- 172.21.15.171:0/1637726043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c8102510 msgr2=0x7f94c8103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:08.337 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c8102510 0x7f94c8103960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d950 tx=0x7f94bc00dc60 comp rx=0 tx=0).stop 2024-03-26T17:59:08.337 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 -- 172.21.15.171:0/1637726043 shutdown_connections 2024-03-26T17:59:08.338 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f94b00a2fb0 0x7f94b00a5470 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:08.338 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f94c8102eb0 0x7f94c8103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:08.338 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94c8101310 0x7f94c8105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:08.338 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 --2- 172.21.15.171:0/1637726043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c8102510 0x7f94c8103960 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:08.338 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 -- 172.21.15.171:0/1637726043 >> 172.21.15.171:0/1637726043 conn(0x7f94c80fc8d0 msgr2=0x7f94c806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:08.338 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 -- 172.21.15.171:0/1637726043 shutdown_connections 2024-03-26T17:59:08.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-03-26T17:59:08.353+0000 7f94ce472700 1 -- 172.21.15.171:0/1637726043 wait complete. 2024-03-26T17:59:09.017 DEBUG:teuthology.orchestra.run.smithi049:mds.a> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.a.service 2024-03-26T17:59:09.020 DEBUG:teuthology.orchestra.run.smithi049:mds.d> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.d.service 2024-03-26T17:59:09.063 DEBUG:teuthology.orchestra.run.smithi049:mds.g> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.g.service 2024-03-26T17:59:09.105 DEBUG:teuthology.orchestra.run.smithi049:mds.j> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.j.service 2024-03-26T17:59:09.149 DEBUG:teuthology.orchestra.run.smithi115:mds.b> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.b.service 2024-03-26T17:59:09.151 DEBUG:teuthology.orchestra.run.smithi115:mds.e> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.e.service 2024-03-26T17:59:09.195 DEBUG:teuthology.orchestra.run.smithi115:mds.h> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.h.service 2024-03-26T17:59:09.237 DEBUG:teuthology.orchestra.run.smithi115:mds.k> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.k.service 2024-03-26T17:59:09.281 DEBUG:teuthology.orchestra.run.smithi171:mds.c> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.c.service 2024-03-26T17:59:09.284 DEBUG:teuthology.orchestra.run.smithi171:mds.f> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.f.service 2024-03-26T17:59:09.327 DEBUG:teuthology.orchestra.run.smithi171:mds.i> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.i.service 2024-03-26T17:59:09.370 DEBUG:teuthology.orchestra.run.smithi171:mds.l> sudo journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.l.service 2024-03-26T17:59:09.413 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-26T17:59:09.414 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:59:09.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:09 smithi115 ceph-mon[36174]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:59:09.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:09 smithi115 ceph-mon[36174]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-03-26T17:59:09.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:09.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:09.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:09.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:09.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:09.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:09.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:09.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:09 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:09.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.544+0000 7f7397ae8640 1 -- 172.21.15.49:0/1766514229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7390102390 msgr2=0x7f7390108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:09.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.544+0000 7f7397ae8640 1 --2- 172.21.15.49:0/1766514229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7390102390 0x7f7390108c20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f738c009ca0 tx=0x7f738c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.544+0000 7f7397ae8640 1 -- 172.21.15.49:0/1766514229 shutdown_connections 2024-03-26T17:59:09.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.544+0000 7f7397ae8640 1 --2- 172.21.15.49:0/1766514229 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7390102390 0x7f7390108c20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.544+0000 7f7397ae8640 1 --2- 172.21.15.49:0/1766514229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73901019d0 0x7f7390101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.544+0000 7f7397ae8640 1 --2- 172.21.15.49:0/1766514229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73901007e0 0x7f7390100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.544+0000 7f7397ae8640 1 -- 172.21.15.49:0/1766514229 >> 172.21.15.49:0/1766514229 conn(0x7f73900fbd50 msgr2=0x7f73900fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.544+0000 7f7397ae8640 1 -- 172.21.15.49:0/1766514229 shutdown_connections 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.544+0000 7f7397ae8640 1 -- 172.21.15.49:0/1766514229 wait complete. 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.544+0000 7f7397ae8640 1 Processor -- start 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7397ae8640 1 -- start start 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7397ae8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73901007e0 0x7f7390197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7397ae8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73901019d0 0x7f73901980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7397ae8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7390102390 0x7f739019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7397ae8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f739010d580 con 0x7f73901019d0 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7397ae8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f739010d400 con 0x7f73901007e0 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7397ae8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f739010d700 con 0x7f7390102390 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395083640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73901019d0 0x7f73901980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395083640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73901019d0 0x7f73901980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47042/0 (socket says 172.21.15.49:47042) 2024-03-26T17:59:09.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7396085640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7390102390 0x7f739019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395083640 1 -- 172.21.15.49:0/2979910802 learned_addr learned my addr 172.21.15.49:0/2979910802 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395884640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73901007e0 0x7f7390197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395083640 1 -- 172.21.15.49:0/2979910802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7390102390 msgr2=0x7f739019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395083640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7390102390 0x7f739019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395083640 1 -- 172.21.15.49:0/2979910802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73901007e0 msgr2=0x7f7390197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395083640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73901007e0 0x7f7390197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395083640 1 -- 172.21.15.49:0/2979910802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739019f790 con 0x7f73901019d0 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7396085640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7390102390 0x7f739019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395884640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73901007e0 0x7f7390197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.545+0000 7f7395083640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73901019d0 0x7f73901980b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f738000d950 tx=0x7f738000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.546+0000 7f7386ffd640 1 -- 172.21.15.49:0/2979910802 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380017030 con 0x7f73901019d0 2024-03-26T17:59:09.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.546+0000 7f7397ae8640 1 -- 172.21.15.49:0/2979910802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739019fa80 con 0x7f73901019d0 2024-03-26T17:59:09.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.546+0000 7f7386ffd640 1 -- 172.21.15.49:0/2979910802 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7380004020 con 0x7f73901019d0 2024-03-26T17:59:09.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.546+0000 7f7386ffd640 1 -- 172.21.15.49:0/2979910802 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73800047a0 con 0x7f73901019d0 2024-03-26T17:59:09.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.546+0000 7f7397ae8640 1 -- 172.21.15.49:0/2979910802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739019ffd0 con 0x7f73901019d0 2024-03-26T17:59:09.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.547+0000 7f7397ae8640 1 -- 172.21.15.49:0/2979910802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7390107b00 con 0x7f73901019d0 2024-03-26T17:59:09.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.548+0000 7f7386ffd640 1 -- 172.21.15.49:0/2979910802 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f73800041d0 con 0x7f73901019d0 2024-03-26T17:59:09.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.548+0000 7f7386ffd640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f737c0a2f20 0x7f737c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:09.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.548+0000 7f7395884640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f737c0a2f20 0x7f737c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:09.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.548+0000 7f7386ffd640 1 -- 172.21.15.49:0/2979910802 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f73800ce930 con 0x7f73901019d0 2024-03-26T17:59:09.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.549+0000 7f7395884640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f737c0a2f20 0x7f737c0a53e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7390101830 tx=0x7f7388005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:09.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.553+0000 7f7386ffd640 1 -- 172.21.15.49:0/2979910802 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f738009bc00 con 0x7f73901019d0 2024-03-26T17:59:09.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:09 smithi171 ceph-mon[36033]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:59:09.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:09 smithi171 ceph-mon[36033]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-03-26T17:59:09.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:09.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:09.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:09.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:09.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:09.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:09.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:09.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:09 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:09.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.738+0000 7f7397ae8640 1 -- 172.21.15.49:0/2979910802 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f739002d420 con 0x7f73901019d0 2024-03-26T17:59:09.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.739+0000 7f7386ffd640 1 -- 172.21.15.49:0/2979910802 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18464 (secure 0 0 0) 0x7f738009b500 con 0x7f73901019d0 2024-03-26T17:59:09.723 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:09.723 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:00.651658+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:59:09.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 -- 172.21.15.49:0/2979910802 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f737c0a2f20 msgr2=0x7f737c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:09.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f737c0a2f20 0x7f737c0a53e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7390101830 tx=0x7f7388005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:09.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 -- 172.21.15.49:0/2979910802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73901019d0 msgr2=0x7f73901980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:09.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73901019d0 0x7f73901980b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f738000d950 tx=0x7f738000de10 comp rx=0 tx=0).stop 2024-03-26T17:59:09.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 -- 172.21.15.49:0/2979910802 shutdown_connections 2024-03-26T17:59:09.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f737c0a2f20 0x7f737c0a53e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7390102390 0x7f739019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73901019d0 0x7f73901980b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 --2- 172.21.15.49:0/2979910802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73901007e0 0x7f7390197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 -- 172.21.15.49:0/2979910802 >> 172.21.15.49:0/2979910802 conn(0x7f73900fbd50 msgr2=0x7f7390104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:09.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 -- 172.21.15.49:0/2979910802 shutdown_connections 2024-03-26T17:59:09.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.741+0000 7f7397ae8640 1 -- 172.21.15.49:0/2979910802 wait complete. 2024-03-26T17:59:09.734 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:09 smithi049 ceph-mon[32331]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:59:09.735 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:09 smithi049 ceph-mon[32331]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-03-26T17:59:09.735 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:09.735 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:09.735 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:09.735 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:09.735 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:09.736 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:09.736 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:09.736 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:09 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:09.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:59:09.739 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:59:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.866+0000 7f61b70a1640 1 -- 172.21.15.49:0/961524107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b0102390 msgr2=0x7f61b0108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.866+0000 7f61b70a1640 1 --2- 172.21.15.49:0/961524107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b0102390 0x7f61b0108c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009ca0 tx=0x7f61a802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.866+0000 7f61b70a1640 1 -- 172.21.15.49:0/961524107 shutdown_connections 2024-03-26T17:59:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.866+0000 7f61b70a1640 1 --2- 172.21.15.49:0/961524107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61b01019d0 0x7f61b0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.866+0000 7f61b70a1640 1 --2- 172.21.15.49:0/961524107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b0102390 0x7f61b0108c20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.866+0000 7f61b70a1640 1 --2- 172.21.15.49:0/961524107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61b01007e0 0x7f61b0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.866+0000 7f61b70a1640 1 -- 172.21.15.49:0/961524107 >> 172.21.15.49:0/961524107 conn(0x7f61b00fbd70 msgr2=0x7f61b00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.866+0000 7f61b70a1640 1 -- 172.21.15.49:0/961524107 shutdown_connections 2024-03-26T17:59:09.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.866+0000 7f61b70a1640 1 -- 172.21.15.49:0/961524107 wait complete. 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61b70a1640 1 Processor -- start 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61b70a1640 1 -- start start 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61b70a1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61b01007e0 0x7f61b0197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61b70a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b01019d0 0x7f61b0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61b70a1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61b0102390 0x7f61b019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61b70a1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f61b010d590 con 0x7f61b01019d0 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61b70a1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f61b010d410 con 0x7f61b01007e0 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61b70a1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f61b010d710 con 0x7f61b0102390 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b01019d0 0x7f61b0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b01019d0 0x7f61b0198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47060/0 (socket says 172.21.15.49:47060) 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61affff640 1 -- 172.21.15.49:0/288610372 learned_addr learned my addr 172.21.15.49:0/288610372 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:09.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61b4e3d640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61b01007e0 0x7f61b0197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.867+0000 7f61b563e640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61b0102390 0x7f61b019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61affff640 1 -- 172.21.15.49:0/288610372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61b0102390 msgr2=0x7f61b019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61affff640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61b0102390 0x7f61b019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61affff640 1 -- 172.21.15.49:0/288610372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61b01007e0 msgr2=0x7f61b0197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61affff640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61b01007e0 0x7f61b0197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61affff640 1 -- 172.21.15.49:0/288610372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b019f810 con 0x7f61b01019d0 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61affff640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b01019d0 0x7f61b0198130 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9f0 tx=0x7f619c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61adffb640 1 -- 172.21.15.49:0/288610372 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c00cd80 con 0x7f61b01019d0 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61adffb640 1 -- 172.21.15.49:0/288610372 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f619c005500 con 0x7f61b01019d0 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61b70a1640 1 -- 172.21.15.49:0/288610372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b019fb00 con 0x7f61b01019d0 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61adffb640 1 -- 172.21.15.49:0/288610372 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c010690 con 0x7f61b01019d0 2024-03-26T17:59:09.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.868+0000 7f61b70a1640 1 -- 172.21.15.49:0/288610372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b01a0000 con 0x7f61b01019d0 2024-03-26T17:59:09.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.869+0000 7f61b70a1640 1 -- 172.21.15.49:0/288610372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b0107aa0 con 0x7f61b01019d0 2024-03-26T17:59:09.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.870+0000 7f61adffb640 1 -- 172.21.15.49:0/288610372 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f619c0046c0 con 0x7f61b01019d0 2024-03-26T17:59:09.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.871+0000 7f61adffb640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61a00a2f20 0x7f61a00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:09.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.871+0000 7f61b4e3d640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61a00a2f20 0x7f61a00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:09.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.871+0000 7f61adffb640 1 -- 172.21.15.49:0/288610372 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f619c0cf310 con 0x7f61b01019d0 2024-03-26T17:59:09.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.871+0000 7f61b4e3d640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61a00a2f20 0x7f61a00a53e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f61b0101830 tx=0x7f61a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:09.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:09.875+0000 7f61adffb640 1 -- 172.21.15.49:0/288610372 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f619c09c5e0 con 0x7f61b01019d0 2024-03-26T17:59:10.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.059+0000 7f61b70a1640 1 -- 172.21.15.49:0/288610372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61b002d3f0 con 0x7f61b01019d0 2024-03-26T17:59:10.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.059+0000 7f61adffb640 1 -- 172.21.15.49:0/288610372 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f619c0bc0d0 con 0x7f61b01019d0 2024-03-26T17:59:10.042 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T17:59:10.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 -- 172.21.15.49:0/288610372 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61a00a2f20 msgr2=0x7f61a00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:10.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61a00a2f20 0x7f61a00a53e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f61b0101830 tx=0x7f61a4005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:10.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 -- 172.21.15.49:0/288610372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b01019d0 msgr2=0x7f61b0198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:10.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b01019d0 0x7f61b0198130 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9f0 tx=0x7f619c00edf0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 -- 172.21.15.49:0/288610372 shutdown_connections 2024-03-26T17:59:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61a00a2f20 0x7f61a00a53e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61b0102390 0x7f61b019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61b01019d0 0x7f61b0198130 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 --2- 172.21.15.49:0/288610372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61b01007e0 0x7f61b0197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 -- 172.21.15.49:0/288610372 >> 172.21.15.49:0/288610372 conn(0x7f61b00fbd70 msgr2=0x7f61b0104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 -- 172.21.15.49:0/288610372 shutdown_connections 2024-03-26T17:59:10.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.061+0000 7f61b70a1640 1 -- 172.21.15.49:0/288610372 wait complete. 2024-03-26T17:59:10.056 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:10.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 -- 172.21.15.49:0/3652506521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc94102ad0 msgr2=0x7fbc94102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 --2- 172.21.15.49:0/3652506521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc94102ad0 0x7fbc94102f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009900 tx=0x7fbc8002bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 -- 172.21.15.49:0/3652506521 shutdown_connections 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 --2- 172.21.15.49:0/3652506521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc94103490 0x7fbc94109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 --2- 172.21.15.49:0/3652506521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc94102ad0 0x7fbc94102f50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 --2- 172.21.15.49:0/3652506521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc941018e0 0x7fbc94101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 -- 172.21.15.49:0/3652506521 >> 172.21.15.49:0/3652506521 conn(0x7fbc940fce70 msgr2=0x7fbc940ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 -- 172.21.15.49:0/3652506521 shutdown_connections 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 -- 172.21.15.49:0/3652506521 wait complete. 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 Processor -- start 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 -- start start 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc941018e0 0x7fbc94197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc94102ad0 0x7fbc94198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:10.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc94103490 0x7fbc9419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9410d560 con 0x7fbc94103490 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9410d3e0 con 0x7fbc94102ad0 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.181+0000 7fbc9acdc640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9410d6e0 con 0x7fbc941018e0 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc99279640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc94103490 0x7fbc9419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc98a78640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc941018e0 0x7fbc94197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc99279640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc94103490 0x7fbc9419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47092/0 (socket says 172.21.15.49:47092) 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc99279640 1 -- 172.21.15.49:0/2808922257 learned_addr learned my addr 172.21.15.49:0/2808922257 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc93fff640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc94102ad0 0x7fbc94198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc99279640 1 -- 172.21.15.49:0/2808922257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc941018e0 msgr2=0x7fbc94197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc99279640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc941018e0 0x7fbc94197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc99279640 1 -- 172.21.15.49:0/2808922257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc94102ad0 msgr2=0x7fbc94198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc99279640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc94102ad0 0x7fbc94198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc99279640 1 -- 172.21.15.49:0/2808922257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9419f840 con 0x7fbc94103490 2024-03-26T17:59:10.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc99279640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc94103490 0x7fbc9419f1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00cad0 tx=0x7fbc8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc91ffb640 1 -- 172.21.15.49:0/2808922257 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00ae20 con 0x7fbc94103490 2024-03-26T17:59:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc91ffb640 1 -- 172.21.15.49:0/2808922257 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c005020 con 0x7fbc94103490 2024-03-26T17:59:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc9acdc640 1 -- 172.21.15.49:0/2808922257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9419fb30 con 0x7fbc94103490 2024-03-26T17:59:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.182+0000 7fbc91ffb640 1 -- 172.21.15.49:0/2808922257 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00f6f0 con 0x7fbc94103490 2024-03-26T17:59:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.183+0000 7fbc9acdc640 1 -- 172.21.15.49:0/2808922257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc941a0080 con 0x7fbc94103490 2024-03-26T17:59:10.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.183+0000 7fbc9acdc640 1 -- 172.21.15.49:0/2808922257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc94108c80 con 0x7fbc94103490 2024-03-26T17:59:10.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.184+0000 7fbc91ffb640 1 -- 172.21.15.49:0/2808922257 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fbc8c0051d0 con 0x7fbc94103490 2024-03-26T17:59:10.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.185+0000 7fbc91ffb640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbc840a2f20 0x7fbc840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:10.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.185+0000 7fbc98a78640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbc840a2f20 0x7fbc840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:10.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.185+0000 7fbc91ffb640 1 -- 172.21.15.49:0/2808922257 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbc8c0cf2c0 con 0x7fbc94103490 2024-03-26T17:59:10.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.185+0000 7fbc98a78640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbc840a2f20 0x7fbc840a53e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbc94199100 tx=0x7fbc8800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:10.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.189+0000 7fbc91ffb640 1 -- 172.21.15.49:0/2808922257 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbc8c014030 con 0x7fbc94103490 2024-03-26T17:59:10.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.411+0000 7fbc9acdc640 1 -- 172.21.15.49:0/2808922257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc9402d4c0 con 0x7fbc94103490 2024-03-26T17:59:10.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.411+0000 7fbc91ffb640 1 -- 172.21.15.49:0/2808922257 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fbc8c09bf40 con 0x7fbc94103490 2024-03-26T17:59:10.395 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:10.395 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-26T17:59:10.395 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 1 2024-03-26T17:59:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 -- 172.21.15.49:0/2808922257 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbc840a2f20 msgr2=0x7fbc840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbc840a2f20 0x7fbc840a53e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbc94199100 tx=0x7fbc8800a2d0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 -- 172.21.15.49:0/2808922257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc94103490 msgr2=0x7fbc9419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc94103490 0x7fbc9419f1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00cad0 tx=0x7fbc8c00cf90 comp rx=0 tx=0).stop 2024-03-26T17:59:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 -- 172.21.15.49:0/2808922257 shutdown_connections 2024-03-26T17:59:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbc840a2f20 0x7fbc840a53e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc94103490 0x7fbc9419f1a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc94102ad0 0x7fbc94198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 --2- 172.21.15.49:0/2808922257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc941018e0 0x7fbc94197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 -- 172.21.15.49:0/2808922257 >> 172.21.15.49:0/2808922257 conn(0x7fbc940fce70 msgr2=0x7fbc94105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 -- 172.21.15.49:0/2808922257 shutdown_connections 2024-03-26T17:59:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:10.414+0000 7fbc9acdc640 1 -- 172.21.15.49:0/2808922257 wait complete. 2024-03-26T17:59:10.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:10 smithi171 ceph-mon[36033]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:10.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:10 smithi171 ceph-mon[36033]: Deploying daemon mds.l on smithi171 2024-03-26T17:59:10.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:10 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2979910802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:10.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:10 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/288610372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:59:10.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:10 smithi115 ceph-mon[36174]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:10.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:10 smithi115 ceph-mon[36174]: Deploying daemon mds.l on smithi171 2024-03-26T17:59:10.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:10 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2979910802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:10.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:10 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/288610372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:59:10.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:10 smithi049 ceph-mon[32331]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:10.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:10 smithi049 ceph-mon[32331]: Deploying daemon mds.l on smithi171 2024-03-26T17:59:10.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:10 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2979910802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:10.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:10 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/288610372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:59:11.566 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:11 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2808922257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:11.566 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 17:59:11 smithi171 systemd[1]: Starting Ceph mds.l for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:59:11.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:11 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2808922257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:11.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:11 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2808922257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:11.835 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 17:59:11 smithi171 podman[66163]: 2024-03-26 17:59:11.486494557 +0000 UTC m=+0.017966349 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:59:11.835 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 17:59:11 smithi171 podman[66163]: 2024-03-26 17:59:11.598091107 +0000 UTC m=+0.129562923 container create b3e097598a091b0729a802929fabd95cf7d2858a9bf41d9f11150c6a7a37a560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-26T17:59:11.835 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 17:59:11 smithi171 podman[66163]: 2024-03-26 17:59:11.826238761 +0000 UTC m=+0.357710537 container init b3e097598a091b0729a802929fabd95cf7d2858a9bf41d9f11150c6a7a37a560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS) 2024-03-26T17:59:11.835 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 17:59:11 smithi171 podman[66163]: 2024-03-26 17:59:11.829071717 +0000 UTC m=+0.360543495 container start b3e097598a091b0729a802929fabd95cf7d2858a9bf41d9f11150c6a7a37a560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-26T17:59:12.107 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 17:59:11 smithi171 ceph-mds[66181]: -- 172.21.15.171:0/4132680686 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a1d2cda40 con 0x555a1d2a0800 2024-03-26T17:59:12.107 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 17:59:11 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l[66177]: starting mds.l at 2024-03-26T17:59:12.107 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 17:59:11 smithi171 bash[66163]: b3e097598a091b0729a802929fabd95cf7d2858a9bf41d9f11150c6a7a37a560 2024-03-26T17:59:12.107 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 17:59:11 smithi171 systemd[1]: Started Ceph mds.l for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:12.409 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:12.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.546+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/124014461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1f4107640 msgr2=0x7fa1f4109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.546+0000 7fa1fa6e8640 1 --2- 172.21.15.49:0/124014461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1f4107640 0x7fa1f4109a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009ca0 tx=0x7fa1ec02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.546+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/124014461 shutdown_connections 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.546+0000 7fa1fa6e8640 1 --2- 172.21.15.49:0/124014461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1f4107640 0x7fa1f4109a90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.546+0000 7fa1fa6e8640 1 --2- 172.21.15.49:0/124014461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1f4104d10 0x7fa1f4107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.546+0000 7fa1fa6e8640 1 --2- 172.21.15.49:0/124014461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1f41023e0 0x7fa1f41047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.546+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/124014461 >> 172.21.15.49:0/124014461 conn(0x7fa1f40fbdf0 msgr2=0x7fa1f40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.546+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/124014461 shutdown_connections 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.546+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/124014461 wait complete. 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1fa6e8640 1 Processor -- start 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1fa6e8640 1 -- start start 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1fa6e8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1f41023e0 0x7fa1f4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1fa6e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1f4104d10 0x7fa1f41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:12.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1fa6e8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1f4107640 0x7fa1f419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1fa6e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f410d590 con 0x7fa1f4104d10 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1fa6e8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f410d410 con 0x7fa1f41023e0 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1fa6e8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f410d710 con 0x7fa1f4107640 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1f4104d10 0x7fa1f41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1f37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1f4104d10 0x7fa1f41980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47106/0 (socket says 172.21.15.49:47106) 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1f37fe640 1 -- 172.21.15.49:0/535878214 learned_addr learned my addr 172.21.15.49:0/535878214 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1f8c85640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1f4107640 0x7fa1f419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.547+0000 7fa1f3fff640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1f41023e0 0x7fa1f4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1f37fe640 1 -- 172.21.15.49:0/535878214 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1f4107640 msgr2=0x7fa1f419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1f37fe640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1f4107640 0x7fa1f419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1f37fe640 1 -- 172.21.15.49:0/535878214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1f41023e0 msgr2=0x7fa1f4197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:12.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1f37fe640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1f41023e0 0x7fa1f4197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:12.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1f37fe640 1 -- 172.21.15.49:0/535878214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f419f7b0 con 0x7fa1f4104d10 2024-03-26T17:59:12.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1f3fff640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1f41023e0 0x7fa1f4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:12.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1f37fe640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1f4104d10 0x7fa1f41980d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800d770 tx=0x7fa1e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:12.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1f17fa640 1 -- 172.21.15.49:0/535878214 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8017070 con 0x7fa1f4104d10 2024-03-26T17:59:12.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/535878214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f419faa0 con 0x7fa1f4104d10 2024-03-26T17:59:12.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1f17fa640 1 -- 172.21.15.49:0/535878214 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e8009c30 con 0x7fa1f4104d10 2024-03-26T17:59:12.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1f17fa640 1 -- 172.21.15.49:0/535878214 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800f620 con 0x7fa1f4104d10 2024-03-26T17:59:12.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.548+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/535878214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f419fff0 con 0x7fa1f4104d10 2024-03-26T17:59:12.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.549+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/535878214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f40fd8e0 con 0x7fa1f4104d10 2024-03-26T17:59:12.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.550+0000 7fa1f17fa640 1 -- 172.21.15.49:0/535878214 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fa1e801a070 con 0x7fa1f4104d10 2024-03-26T17:59:12.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.551+0000 7fa1f17fa640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa1dc0a2f20 0x7fa1dc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:12.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.551+0000 7fa1f3fff640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa1dc0a2f20 0x7fa1dc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:12.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.551+0000 7fa1f17fa640 1 -- 172.21.15.49:0/535878214 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1e80cea40 con 0x7fa1f4104d10 2024-03-26T17:59:12.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.551+0000 7fa1f3fff640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa1dc0a2f20 0x7fa1dc0a53e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009c20 tx=0x7fa1e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.556+0000 7fa1f17fa640 1 -- 172.21.15.49:0/535878214 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa1e8009de0 con 0x7fa1f4104d10 2024-03-26T17:59:12.606 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:12 smithi115 ceph-mon[36174]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:12.606 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:12 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] up:boot 2024-03-26T17:59:12.606 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:12 smithi115 ceph-mon[36174]: fsmap 1 up:standby 2024-03-26T17:59:12.606 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T17:59:12.606 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:12.607 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:12.607 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:12.607 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:12.607 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:12.607 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:12 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:12.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:12 smithi171 ceph-mon[36033]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:12.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:12 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] up:boot 2024-03-26T17:59:12.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:12 smithi171 ceph-mon[36033]: fsmap 1 up:standby 2024-03-26T17:59:12.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T17:59:12.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:12.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:12.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:12.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:12.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:12.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:12 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:12.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:12 smithi049 ceph-mon[32331]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:12.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:12 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] up:boot 2024-03-26T17:59:12.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:12 smithi049 ceph-mon[32331]: fsmap 1 up:standby 2024-03-26T17:59:12.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T17:59:12.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:12.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:12.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:12.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:12.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:12.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:12 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.779+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/535878214 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1f402d460 con 0x7fa1f4104d10 2024-03-26T17:59:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.780+0000 7fa1f17fa640 1 -- 172.21.15.49:0/535878214 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fa1e809bd10 con 0x7fa1f4104d10 2024-03-26T17:59:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:12.763 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-26T17:59:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2024-03-26T17:59:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/535878214 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa1dc0a2f20 msgr2=0x7fa1dc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa1dc0a2f20 0x7fa1dc0a53e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009c20 tx=0x7fa1e4005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/535878214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1f4104d10 msgr2=0x7fa1f41980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1f4104d10 0x7fa1f41980d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800d770 tx=0x7fa1e800dc30 comp rx=0 tx=0).stop 2024-03-26T17:59:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/535878214 shutdown_connections 2024-03-26T17:59:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa1dc0a2f20 0x7fa1dc0a53e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa1f4107640 0x7fa1f419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1f4104d10 0x7fa1f41980d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 --2- 172.21.15.49:0/535878214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1f41023e0 0x7fa1f4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/535878214 >> 172.21.15.49:0/535878214 conn(0x7fa1f40fbdf0 msgr2=0x7fa1f4102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/535878214 shutdown_connections 2024-03-26T17:59:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:12.782+0000 7fa1fa6e8640 1 -- 172.21.15.49:0/535878214 wait complete. 2024-03-26T17:59:13.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:13 smithi171 ceph-mon[36033]: Deploying daemon mds.k on smithi115 2024-03-26T17:59:13.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:13 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/535878214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:13.631 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:13 smithi115 ceph-mon[36174]: Deploying daemon mds.k on smithi115 2024-03-26T17:59:13.632 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:13 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/535878214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:13.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:13 smithi049 ceph-mon[32331]: Deploying daemon mds.k on smithi115 2024-03-26T17:59:13.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:13 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/535878214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:14.355 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 17:59:14 smithi115 systemd[1]: Starting Ceph mds.k for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:59:14.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:14 smithi171 ceph-mon[36033]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:14.624 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:14 smithi115 ceph-mon[36174]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:14.624 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 17:59:14 smithi115 podman[65512]: 2024-03-26 17:59:14.296850068 +0000 UTC m=+0.017416970 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:59:14.625 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 17:59:14 smithi115 podman[65512]: 2024-03-26 17:59:14.406126182 +0000 UTC m=+0.126693067 container create d6dc72aac7202ced1c8f012cc9d1850ddc8386970966ca8507f7981f7e0bef12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5) 2024-03-26T17:59:14.625 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 17:59:14 smithi115 podman[65512]: 2024-03-26 17:59:14.642096163 +0000 UTC m=+0.362663052 container init d6dc72aac7202ced1c8f012cc9d1850ddc8386970966ca8507f7981f7e0bef12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-26T17:59:14.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:14 smithi049 ceph-mon[32331]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:14.777 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:14.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.918+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/1467704643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b68104d10 msgr2=0x7f3b68107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:14.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.918+0000 7f3b6fe0b640 1 --2- 172.21.15.49:0/1467704643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b68104d10 0x7f3b68107100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009900 tx=0x7f3b5802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:14.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.918+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/1467704643 shutdown_connections 2024-03-26T17:59:14.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.918+0000 7f3b6fe0b640 1 --2- 172.21.15.49:0/1467704643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b68107640 0x7f3b68109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:14.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.918+0000 7f3b6fe0b640 1 --2- 172.21.15.49:0/1467704643 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b68104d10 0x7f3b68107100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:14.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.918+0000 7f3b6fe0b640 1 --2- 172.21.15.49:0/1467704643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b681023e0 0x7f3b681047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:14.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.918+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/1467704643 >> 172.21.15.49:0/1467704643 conn(0x7f3b680fbd50 msgr2=0x7f3b680fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:14.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/1467704643 shutdown_connections 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/1467704643 wait complete. 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6fe0b640 1 Processor -- start 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6fe0b640 1 -- start start 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6fe0b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b681023e0 0x7f3b68199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6fe0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b68104d10 0x7f3b6819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6fe0b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b68107640 0x7f3b681a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6fe0b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6810d560 con 0x7f3b68104d10 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6fe0b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6810d3e0 con 0x7f3b681023e0 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6fe0b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6810d6e0 con 0x7f3b68107640 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6d3a6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b68104d10 0x7f3b6819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:14.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6d3a6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b68104d10 0x7f3b6819a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47130/0 (socket says 172.21.15.49:47130) 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.919+0000 7f3b6d3a6640 1 -- 172.21.15.49:0/322016408 learned_addr learned my addr 172.21.15.49:0/322016408 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6dba7640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b681023e0 0x7f3b68199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6e3a8640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b68107640 0x7f3b681a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6d3a6640 1 -- 172.21.15.49:0/322016408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b68107640 msgr2=0x7f3b681a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6d3a6640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b68107640 0x7f3b681a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6d3a6640 1 -- 172.21.15.49:0/322016408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b681023e0 msgr2=0x7f3b68199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6d3a6640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b681023e0 0x7f3b68199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6d3a6640 1 -- 172.21.15.49:0/322016408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b681a1a10 con 0x7f3b68104d10 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6dba7640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b681023e0 0x7f3b68199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6e3a8640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b68107640 0x7f3b681a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6d3a6640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b68104d10 0x7f3b6819a330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b58005ab0 tx=0x7f3b58005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:14.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b5effd640 1 -- 172.21.15.49:0/322016408 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58041070 con 0x7f3b68104d10 2024-03-26T17:59:14.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b5effd640 1 -- 172.21.15.49:0/322016408 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b580054e0 con 0x7f3b68104d10 2024-03-26T17:59:14.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/322016408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b681a1ca0 con 0x7f3b68104d10 2024-03-26T17:59:14.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b5effd640 1 -- 172.21.15.49:0/322016408 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58034440 con 0x7f3b68104d10 2024-03-26T17:59:14.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.920+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/322016408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b681a2190 con 0x7f3b68104d10 2024-03-26T17:59:14.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.921+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/322016408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b68109130 con 0x7f3b68104d10 2024-03-26T17:59:14.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.923+0000 7f3b5effd640 1 -- 172.21.15.49:0/322016408 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f3b58044070 con 0x7f3b68104d10 2024-03-26T17:59:14.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.924+0000 7f3b5effd640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3b540a2f20 0x7f3b540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:14.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.924+0000 7f3b5effd640 1 -- 172.21.15.49:0/322016408 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3b580f0c60 con 0x7f3b68104d10 2024-03-26T17:59:14.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.924+0000 7f3b6dba7640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3b540a2f20 0x7f3b540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:14.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.924+0000 7f3b6dba7640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3b540a2f20 0x7f3b540a53e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3b640058c0 tx=0x7f3b64005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:14.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:14.929+0000 7f3b5effd640 1 -- 172.21.15.49:0/322016408 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3b580bdf30 con 0x7f3b68104d10 2024-03-26T17:59:14.988 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 17:59:14 smithi115 podman[65512]: 2024-03-26 17:59:14.644973347 +0000 UTC m=+0.365540233 container start d6dc72aac7202ced1c8f012cc9d1850ddc8386970966ca8507f7981f7e0bef12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-03-26T17:59:14.988 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 17:59:14 smithi115 ceph-mds[65531]: -- 172.21.15.115:0/3534481295 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555938933a40 con 0x555938907000 2024-03-26T17:59:14.988 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 17:59:14 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k[65527]: starting mds.k at 2024-03-26T17:59:14.988 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 17:59:14 smithi115 bash[65512]: d6dc72aac7202ced1c8f012cc9d1850ddc8386970966ca8507f7981f7e0bef12 2024-03-26T17:59:14.988 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 17:59:14 smithi115 systemd[1]: Started Ceph mds.k for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:15.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.153+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/322016408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b6802d470 con 0x7f3b68104d10 2024-03-26T17:59:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.154+0000 7f3b5effd640 1 -- 172.21.15.49:0/322016408 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f3b580bd830 con 0x7f3b68104d10 2024-03-26T17:59:15.137 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:15.137 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-26T17:59:15.137 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2024-03-26T17:59:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/322016408 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3b540a2f20 msgr2=0x7f3b540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3b540a2f20 0x7f3b540a53e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3b640058c0 tx=0x7f3b64005850 comp rx=0 tx=0).stop 2024-03-26T17:59:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/322016408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b68104d10 msgr2=0x7f3b6819a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b68104d10 0x7f3b6819a330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b58005ab0 tx=0x7f3b58005dc0 comp rx=0 tx=0).stop 2024-03-26T17:59:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/322016408 shutdown_connections 2024-03-26T17:59:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3b540a2f20 0x7f3b540a53e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:15.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b68107640 0x7f3b681a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b68104d10 0x7f3b6819a330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 --2- 172.21.15.49:0/322016408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b681023e0 0x7f3b68199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/322016408 >> 172.21.15.49:0/322016408 conn(0x7f3b680fbd50 msgr2=0x7f3b68101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/322016408 shutdown_connections 2024-03-26T17:59:15.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:15.156+0000 7f3b6fe0b640 1 -- 172.21.15.49:0/322016408 wait complete. 2024-03-26T17:59:16.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: Deploying daemon mds.j on smithi049 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/322016408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] up:boot 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: fsmap 2 up:standby 2024-03-26T17:59:16.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:15 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T17:59:16.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:16.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:16.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:16.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:16.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:16.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:16.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:16.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: Deploying daemon mds.j on smithi049 2024-03-26T17:59:16.139 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/322016408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:16.139 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] up:boot 2024-03-26T17:59:16.139 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: fsmap 2 up:standby 2024-03-26T17:59:16.139 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:15 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T17:59:16.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:16.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:16.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:16.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:16.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:16.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:16.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:16.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: Deploying daemon mds.j on smithi049 2024-03-26T17:59:16.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/322016408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:16.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] up:boot 2024-03-26T17:59:16.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: fsmap 2 up:standby 2024-03-26T17:59:16.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:15 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T17:59:17.151 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:17.181 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 17:59:17 smithi049 systemd[1]: Starting Ceph mds.j for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:59:17.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.285+0000 7f2ec724b640 1 -- 172.21.15.49:0/1787989618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0107620 msgr2=0x7f2ec0109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:17.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.285+0000 7f2ec724b640 1 --2- 172.21.15.49:0/1787989618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0107620 0x7f2ec0109a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009ca0 tx=0x7f2eb802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:17.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.285+0000 7f2ec724b640 1 -- 172.21.15.49:0/1787989618 shutdown_connections 2024-03-26T17:59:17.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.285+0000 7f2ec724b640 1 --2- 172.21.15.49:0/1787989618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0107620 0x7f2ec0109a10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:17.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.285+0000 7f2ec724b640 1 --2- 172.21.15.49:0/1787989618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec0104cf0 0x7f2ec01070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:17.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.285+0000 7f2ec724b640 1 --2- 172.21.15.49:0/1787989618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec01023c0 0x7f2ec01047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:17.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.285+0000 7f2ec724b640 1 -- 172.21.15.49:0/1787989618 >> 172.21.15.49:0/1787989618 conn(0x7f2ec00fbd50 msgr2=0x7f2ec00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:17.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.285+0000 7f2ec724b640 1 -- 172.21.15.49:0/1787989618 shutdown_connections 2024-03-26T17:59:17.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.285+0000 7f2ec724b640 1 -- 172.21.15.49:0/1787989618 wait complete. 2024-03-26T17:59:17.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec724b640 1 Processor -- start 2024-03-26T17:59:17.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec724b640 1 -- start start 2024-03-26T17:59:17.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec724b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec01023c0 0x7f2ec0101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:17.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec724b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec0104cf0 0x7f2ec0100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:17.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec724b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0107620 0x7f2ec01008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:17.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec724b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec010d580 con 0x7f2ec0107620 2024-03-26T17:59:17.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec724b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec010d400 con 0x7f2ec01023c0 2024-03-26T17:59:17.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec724b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec010d700 con 0x7f2ec0104cf0 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec57e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0107620 0x7f2ec01008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec57e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0107620 0x7f2ec01008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47140/0 (socket says 172.21.15.49:47140) 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec57e8640 1 -- 172.21.15.49:0/3890789131 learned_addr learned my addr 172.21.15.49:0/3890789131 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ec4fe7640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec01023c0 0x7f2ec0101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.286+0000 7f2ebffff640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec0104cf0 0x7f2ec0100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ec57e8640 1 -- 172.21.15.49:0/3890789131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec0104cf0 msgr2=0x7f2ec0100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ec57e8640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec0104cf0 0x7f2ec0100370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ec57e8640 1 -- 172.21.15.49:0/3890789131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec01023c0 msgr2=0x7f2ec0101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ec57e8640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec01023c0 0x7f2ec0101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ec57e8640 1 -- 172.21.15.49:0/3890789131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec0101140 con 0x7f2ec0107620 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ec57e8640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0107620 0x7f2ec01008b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8037040 tx=0x7f2eb8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ebdffb640 1 -- 172.21.15.49:0/3890789131 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8041020 con 0x7f2ec0107620 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ebdffb640 1 -- 172.21.15.49:0/3890789131 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb8034d90 con 0x7f2ec0107620 2024-03-26T17:59:17.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ec724b640 1 -- 172.21.15.49:0/3890789131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec01a1cb0 con 0x7f2ec0107620 2024-03-26T17:59:17.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ebdffb640 1 -- 172.21.15.49:0/3890789131 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8039040 con 0x7f2ec0107620 2024-03-26T17:59:17.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.287+0000 7f2ec724b640 1 -- 172.21.15.49:0/3890789131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec01a21b0 con 0x7f2ec0107620 2024-03-26T17:59:17.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.289+0000 7f2ec724b640 1 -- 172.21.15.49:0/3890789131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec00686b0 con 0x7f2ec0107620 2024-03-26T17:59:17.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.289+0000 7f2ebdffb640 1 -- 172.21.15.49:0/3890789131 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f2eb8044020 con 0x7f2ec0107620 2024-03-26T17:59:17.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.290+0000 7f2ebdffb640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2eb00a2f20 0x7f2eb00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:17.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.290+0000 7f2ec4fe7640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2eb00a2f20 0x7f2eb00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:17.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.290+0000 7f2ebdffb640 1 -- 172.21.15.49:0/3890789131 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2eb80f0e30 con 0x7f2ec0107620 2024-03-26T17:59:17.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.291+0000 7f2ec4fe7640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2eb00a2f20 0x7f2eb00a53e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009bd0 tx=0x7f2eb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:17.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.295+0000 7f2ebdffb640 1 -- 172.21.15.49:0/3890789131 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2eb804b020 con 0x7f2ec0107620 2024-03-26T17:59:17.436 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 17:59:17 smithi049 podman[72476]: 2024-03-26 17:59:17.15146492 +0000 UTC m=+0.018129303 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:59:17.436 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 17:59:17 smithi049 podman[72476]: 2024-03-26 17:59:17.305717921 +0000 UTC m=+0.172382352 container create c7eceaac0a995e8be72b1d49b0317d265ef7bda1953df745afbac9f45b9be88a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-26T17:59:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.529+0000 7f2ec724b640 1 -- 172.21.15.49:0/3890789131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ec002d2e0 con 0x7f2ec0107620 2024-03-26T17:59:17.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.530+0000 7f2ebdffb640 1 -- 172.21.15.49:0/3890789131 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f2ec002d2e0 con 0x7f2ec0107620 2024-03-26T17:59:17.513 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:17.513 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-26T17:59:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.531+0000 7f2ec724b640 1 -- 172.21.15.49:0/3890789131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2eb00a2f20 msgr2=0x7f2eb00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2eb00a2f20 0x7f2eb00a53e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009bd0 tx=0x7f2eb4005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 -- 172.21.15.49:0/3890789131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0107620 msgr2=0x7f2ec01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0107620 0x7f2ec01008b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8037040 tx=0x7f2eb8032f20 comp rx=0 tx=0).stop 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 -- 172.21.15.49:0/3890789131 shutdown_connections 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2eb00a2f20 0x7f2eb00a53e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec0104cf0 0x7f2ec0100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec0107620 0x7f2ec01008b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 --2- 172.21.15.49:0/3890789131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec01023c0 0x7f2ec0101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 -- 172.21.15.49:0/3890789131 >> 172.21.15.49:0/3890789131 conn(0x7f2ec00fbd50 msgr2=0x7f2ec0102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:17.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 -- 172.21.15.49:0/3890789131 shutdown_connections 2024-03-26T17:59:17.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:17.532+0000 7f2ec724b640 1 -- 172.21.15.49:0/3890789131 wait complete. 2024-03-26T17:59:17.797 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 17:59:17 smithi049 podman[72476]: 2024-03-26 17:59:17.563001871 +0000 UTC m=+0.429666234 container init c7eceaac0a995e8be72b1d49b0317d265ef7bda1953df745afbac9f45b9be88a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-j, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-03-26T17:59:17.797 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 17:59:17 smithi049 podman[72476]: 2024-03-26 17:59:17.565655372 +0000 UTC m=+0.432319742 container start c7eceaac0a995e8be72b1d49b0317d265ef7bda1953df745afbac9f45b9be88a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T17:59:17.797 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 17:59:17 smithi049 ceph-mds[72536]: -- 172.21.15.49:0/1970151896 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ffd5387a40 con 0x55ffd535ac00 2024-03-26T17:59:17.797 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 17:59:17 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-j[72530]: starting mds.j at 2024-03-26T17:59:17.797 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 17:59:17 smithi049 bash[72476]: c7eceaac0a995e8be72b1d49b0317d265ef7bda1953df745afbac9f45b9be88a 2024-03-26T17:59:17.797 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 17:59:17 smithi049 systemd[1]: Started Ceph mds.j for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:18.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:17 smithi171 ceph-mon[36033]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:18.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:17 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3890789131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:18.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:18.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:18.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:18.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:18.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:18.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:17 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:18.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:17 smithi115 ceph-mon[36174]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:18.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:17 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3890789131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:18.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:18.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:18.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:18.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:18.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:18.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:17 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:18.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:17 smithi049 ceph-mon[32331]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:18.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:17 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3890789131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:18.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:18.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:18.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:18.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:18.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:18.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:17 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:19.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:18 smithi049 ceph-mon[32331]: Deploying daemon mds.d on smithi049 2024-03-26T17:59:19.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:18 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.49:6834/3996463514,v1:172.21.15.49:6835/3996463514] up:boot 2024-03-26T17:59:19.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:18 smithi049 ceph-mon[32331]: fsmap 3 up:standby 2024-03-26T17:59:19.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:18 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T17:59:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:18 smithi171 ceph-mon[36033]: Deploying daemon mds.d on smithi049 2024-03-26T17:59:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:18 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.49:6834/3996463514,v1:172.21.15.49:6835/3996463514] up:boot 2024-03-26T17:59:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:18 smithi171 ceph-mon[36033]: fsmap 3 up:standby 2024-03-26T17:59:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:18 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T17:59:19.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:18 smithi115 ceph-mon[36174]: Deploying daemon mds.d on smithi049 2024-03-26T17:59:19.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:18 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.49:6834/3996463514,v1:172.21.15.49:6835/3996463514] up:boot 2024-03-26T17:59:19.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:18 smithi115 ceph-mon[36174]: fsmap 3 up:standby 2024-03-26T17:59:19.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:18 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T17:59:19.526 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.676+0000 7fe5da17e640 1 -- 172.21.15.49:0/285285798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4104cf0 msgr2=0x7fe5d41070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.676+0000 7fe5da17e640 1 --2- 172.21.15.49:0/285285798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4104cf0 0x7fe5d41070e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009900 tx=0x7fe5c402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.676+0000 7fe5da17e640 1 -- 172.21.15.49:0/285285798 shutdown_connections 2024-03-26T17:59:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.676+0000 7fe5da17e640 1 --2- 172.21.15.49:0/285285798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5d4107620 0x7fe5d4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.676+0000 7fe5da17e640 1 --2- 172.21.15.49:0/285285798 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4104cf0 0x7fe5d41070e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:19.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.676+0000 7fe5da17e640 1 --2- 172.21.15.49:0/285285798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d41023c0 0x7fe5d41047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.676+0000 7fe5da17e640 1 -- 172.21.15.49:0/285285798 >> 172.21.15.49:0/285285798 conn(0x7fe5d40fbd50 msgr2=0x7fe5d40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.676+0000 7fe5da17e640 1 -- 172.21.15.49:0/285285798 shutdown_connections 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.676+0000 7fe5da17e640 1 -- 172.21.15.49:0/285285798 wait complete. 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.676+0000 7fe5da17e640 1 Processor -- start 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5da17e640 1 -- start start 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5da17e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d41023c0 0x7fe5d4101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5da17e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4104cf0 0x7fe5d4100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5da17e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5d4107620 0x7fe5d41008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5da17e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410d560 con 0x7fe5d4104cf0 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5da17e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410d3e0 con 0x7fe5d41023c0 2024-03-26T17:59:19.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5da17e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410d6e0 con 0x7fe5d4107620 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4104cf0 0x7fe5d4100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4104cf0 0x7fe5d4100370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55094/0 (socket says 172.21.15.49:55094) 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d2ffd640 1 -- 172.21.15.49:0/1381111886 learned_addr learned my addr 172.21.15.49:0/1381111886 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d3fff640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5d4107620 0x7fe5d41008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d37fe640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d41023c0 0x7fe5d4101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d2ffd640 1 -- 172.21.15.49:0/1381111886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5d4107620 msgr2=0x7fe5d41008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d2ffd640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5d4107620 0x7fe5d41008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d2ffd640 1 -- 172.21.15.49:0/1381111886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d41023c0 msgr2=0x7fe5d4101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d2ffd640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d41023c0 0x7fe5d4101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d2ffd640 1 -- 172.21.15.49:0/1381111886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d4101140 con 0x7fe5d4104cf0 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.677+0000 7fe5d2ffd640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4104cf0 0x7fe5d4100370 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4005ab0 tx=0x7fe5c4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.678+0000 7fe5d0ff9640 1 -- 172.21.15.49:0/1381111886 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4041070 con 0x7fe5d4104cf0 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.678+0000 7fe5da17e640 1 -- 172.21.15.49:0/1381111886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d41a1cf0 con 0x7fe5d4104cf0 2024-03-26T17:59:19.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.678+0000 7fe5d0ff9640 1 -- 172.21.15.49:0/1381111886 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c40054e0 con 0x7fe5d4104cf0 2024-03-26T17:59:19.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.678+0000 7fe5d0ff9640 1 -- 172.21.15.49:0/1381111886 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4034440 con 0x7fe5d4104cf0 2024-03-26T17:59:19.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.678+0000 7fe5da17e640 1 -- 172.21.15.49:0/1381111886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d41a21d0 con 0x7fe5d4104cf0 2024-03-26T17:59:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.679+0000 7fe5da17e640 1 -- 172.21.15.49:0/1381111886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d40fd840 con 0x7fe5d4104cf0 2024-03-26T17:59:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.680+0000 7fe5d0ff9640 1 -- 172.21.15.49:0/1381111886 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fe5c4044070 con 0x7fe5d4104cf0 2024-03-26T17:59:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.681+0000 7fe5d0ff9640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe5c00a2f20 0x7fe5c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.681+0000 7fe5d37fe640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe5c00a2f20 0x7fe5c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.681+0000 7fe5d0ff9640 1 -- 172.21.15.49:0/1381111886 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe5c40f0d00 con 0x7fe5d4104cf0 2024-03-26T17:59:19.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.681+0000 7fe5d37fe640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe5c00a2f20 0x7fe5c00a53e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005fd0 tx=0x7fe5bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.684+0000 7fe5d0ff9640 1 -- 172.21.15.49:0/1381111886 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe5c40bdfd0 con 0x7fe5d4104cf0 2024-03-26T17:59:19.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.917+0000 7fe5da17e640 1 -- 172.21.15.49:0/1381111886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5d402d240 con 0x7fe5d4104cf0 2024-03-26T17:59:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.918+0000 7fe5d0ff9640 1 -- 172.21.15.49:0/1381111886 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fe5c40bd8d0 con 0x7fe5d4104cf0 2024-03-26T17:59:19.901 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:19.901 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-26T17:59:19.901 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 4 2024-03-26T17:59:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 -- 172.21.15.49:0/1381111886 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe5c00a2f20 msgr2=0x7fe5c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe5c00a2f20 0x7fe5c00a53e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005fd0 tx=0x7fe5bc005850 comp rx=0 tx=0).stop 2024-03-26T17:59:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 -- 172.21.15.49:0/1381111886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4104cf0 msgr2=0x7fe5d4100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4104cf0 0x7fe5d4100370 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4005ab0 tx=0x7fe5c4005dc0 comp rx=0 tx=0).stop 2024-03-26T17:59:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 -- 172.21.15.49:0/1381111886 shutdown_connections 2024-03-26T17:59:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe5c00a2f20 0x7fe5c00a53e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:19.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5d4107620 0x7fe5d41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5d4104cf0 0x7fe5d4100370 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 --2- 172.21.15.49:0/1381111886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d41023c0 0x7fe5d4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 -- 172.21.15.49:0/1381111886 >> 172.21.15.49:0/1381111886 conn(0x7fe5d40fbd50 msgr2=0x7fe5d4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 -- 172.21.15.49:0/1381111886 shutdown_connections 2024-03-26T17:59:19.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:19.920+0000 7fe5da17e640 1 -- 172.21.15.49:0/1381111886 wait complete. 2024-03-26T17:59:20.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:19 smithi171 ceph-mon[36033]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:20.213 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:19 smithi049 ceph-mon[32331]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:20.214 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 17:59:20 smithi049 systemd[1]: Starting Ceph mds.d for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:59:20.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:19 smithi115 ceph-mon[36174]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:20.485 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 17:59:20 smithi049 podman[72863]: 2024-03-26 17:59:20.133429208 +0000 UTC m=+0.018314914 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:59:20.485 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 17:59:20 smithi049 podman[72863]: 2024-03-26 17:59:20.25828247 +0000 UTC m=+0.143168154 container create c008ea39c957a90eedad4b2af4567e73bcb948bf313ac3b60a4ee5d8bc5bc0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-26T17:59:20.485 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 17:59:20 smithi049 podman[72863]: 2024-03-26 17:59:20.502555448 +0000 UTC m=+0.387441133 container init c008ea39c957a90eedad4b2af4567e73bcb948bf313ac3b60a4ee5d8bc5bc0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-26T17:59:20.753 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 17:59:20 smithi049 podman[72863]: 2024-03-26 17:59:20.505380498 +0000 UTC m=+0.390266190 container start c008ea39c957a90eedad4b2af4567e73bcb948bf313ac3b60a4ee5d8bc5bc0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-26T17:59:20.754 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 17:59:20 smithi049 ceph-mds[72882]: -- 172.21.15.49:0/1647867888 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f94d72fa40 con 0x55f94d703000 2024-03-26T17:59:20.754 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 17:59:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-d[72878]: starting mds.d at 2024-03-26T17:59:20.754 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 17:59:20 smithi049 bash[72863]: c008ea39c957a90eedad4b2af4567e73bcb948bf313ac3b60a4ee5d8bc5bc0ef 2024-03-26T17:59:20.754 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 17:59:20 smithi049 systemd[1]: Started Ceph mds.d for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:21.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:20 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1381111886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:21.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:20 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:21.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:20 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:21.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:20 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:21.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:20 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:21.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:20 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:21.108 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:20 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:21.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:20 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1381111886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:21.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:20 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:21.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:20 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:21.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:20 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:21.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:20 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:21.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:20 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:21.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:20 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:21.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:20 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1381111886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:21.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:20 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:21.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:20 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:21.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:20 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:21.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:20 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:21.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:20 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:21.254 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:20 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:21.915 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:21.956 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:21 smithi115 ceph-mon[36174]: Deploying daemon mds.e on smithi115 2024-03-26T17:59:21.956 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:21 smithi115 ceph-mon[36174]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:21.957 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:21 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.49:6836/2438318193,v1:172.21.15.49:6837/2438318193] up:boot 2024-03-26T17:59:21.957 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:21 smithi115 ceph-mon[36174]: fsmap 4 up:standby 2024-03-26T17:59:21.957 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:21 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T17:59:22.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.057+0000 7ff98edc4640 1 -- 172.21.15.49:0/89950585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9881023e0 msgr2=0x7ff9881047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.057+0000 7ff98edc4640 1 --2- 172.21.15.49:0/89950585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9881023e0 0x7ff9881047d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff978009900 tx=0x7ff97802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.057+0000 7ff98edc4640 1 -- 172.21.15.49:0/89950585 shutdown_connections 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.057+0000 7ff98edc4640 1 --2- 172.21.15.49:0/89950585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff988107640 0x7ff988109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.057+0000 7ff98edc4640 1 --2- 172.21.15.49:0/89950585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff988104d10 0x7ff988107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.057+0000 7ff98edc4640 1 --2- 172.21.15.49:0/89950585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9881023e0 0x7ff9881047d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.057+0000 7ff98edc4640 1 -- 172.21.15.49:0/89950585 >> 172.21.15.49:0/89950585 conn(0x7ff9880fbdf0 msgr2=0x7ff9880fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.057+0000 7ff98edc4640 1 -- 172.21.15.49:0/89950585 shutdown_connections 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.057+0000 7ff98edc4640 1 -- 172.21.15.49:0/89950585 wait complete. 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98edc4640 1 Processor -- start 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98edc4640 1 -- start start 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98edc4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9881023e0 0x7ff988101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98edc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff988104d10 0x7ff988100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:22.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98edc4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff988107640 0x7ff9881008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98edc4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff98810d5c0 con 0x7ff988104d10 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98edc4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff98810d440 con 0x7ff988107640 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98edc4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff98810d740 con 0x7ff9881023e0 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff987fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff988104d10 0x7ff988100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff987fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff988104d10 0x7ff988100390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55126/0 (socket says 172.21.15.49:55126) 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff987fff640 1 -- 172.21.15.49:0/487317276 learned_addr learned my addr 172.21.15.49:0/487317276 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98cb60640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9881023e0 0x7ff988101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98d361640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff988107640 0x7ff9881008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff987fff640 1 -- 172.21.15.49:0/487317276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9881023e0 msgr2=0x7ff988101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff987fff640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9881023e0 0x7ff988101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff987fff640 1 -- 172.21.15.49:0/487317276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff988107640 msgr2=0x7ff9881008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff987fff640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff988107640 0x7ff9881008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff987fff640 1 -- 172.21.15.49:0/487317276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff988101160 con 0x7ff988104d10 2024-03-26T17:59:22.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff98d361640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff988107640 0x7ff9881008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:22.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.058+0000 7ff987fff640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff988104d10 0x7ff988100390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff97c005fd0 tx=0x7ff97c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:22.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.059+0000 7ff985ffb640 1 -- 172.21.15.49:0/487317276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c017070 con 0x7ff988104d10 2024-03-26T17:59:22.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.059+0000 7ff985ffb640 1 -- 172.21.15.49:0/487317276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff97c009c30 con 0x7ff988104d10 2024-03-26T17:59:22.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.059+0000 7ff98edc4640 1 -- 172.21.15.49:0/487317276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9881a1c90 con 0x7ff988104d10 2024-03-26T17:59:22.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.059+0000 7ff985ffb640 1 -- 172.21.15.49:0/487317276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c010460 con 0x7ff988104d10 2024-03-26T17:59:22.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.059+0000 7ff98edc4640 1 -- 172.21.15.49:0/487317276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9881a2180 con 0x7ff988104d10 2024-03-26T17:59:22.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.060+0000 7ff98edc4640 1 -- 172.21.15.49:0/487317276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9880fd8e0 con 0x7ff988104d10 2024-03-26T17:59:22.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.061+0000 7ff985ffb640 1 -- 172.21.15.49:0/487317276 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7ff97c003e70 con 0x7ff988104d10 2024-03-26T17:59:22.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.062+0000 7ff985ffb640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff9700a2fa0 0x7ff9700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:22.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.062+0000 7ff98cb60640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff9700a2fa0 0x7ff9700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:22.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.062+0000 7ff985ffb640 1 -- 172.21.15.49:0/487317276 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff97c0cec10 con 0x7ff988104d10 2024-03-26T17:59:22.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.062+0000 7ff98cb60640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff9700a2fa0 0x7ff9700a5460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff97802c240 tx=0x7ff97802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:22.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.066+0000 7ff985ffb640 1 -- 172.21.15.49:0/487317276 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff97c020030 con 0x7ff988104d10 2024-03-26T17:59:22.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:21 smithi171 ceph-mon[36033]: Deploying daemon mds.e on smithi115 2024-03-26T17:59:22.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:21 smithi171 ceph-mon[36033]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:22.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:21 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.49:6836/2438318193,v1:172.21.15.49:6837/2438318193] up:boot 2024-03-26T17:59:22.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:21 smithi171 ceph-mon[36033]: fsmap 4 up:standby 2024-03-26T17:59:22.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:21 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T17:59:22.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:21 smithi049 ceph-mon[32331]: Deploying daemon mds.e on smithi115 2024-03-26T17:59:22.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:21 smithi049 ceph-mon[32331]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:22.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:21 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.49:6836/2438318193,v1:172.21.15.49:6837/2438318193] up:boot 2024-03-26T17:59:22.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:21 smithi049 ceph-mon[32331]: fsmap 4 up:standby 2024-03-26T17:59:22.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:21 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T17:59:22.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.289+0000 7ff98edc4640 1 -- 172.21.15.49:0/487317276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff98802d3f0 con 0x7ff988104d10 2024-03-26T17:59:22.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.290+0000 7ff985ffb640 1 -- 172.21.15.49:0/487317276 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7ff97c09bee0 con 0x7ff988104d10 2024-03-26T17:59:22.273 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:22.273 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-26T17:59:22.273 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 5 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 -- 172.21.15.49:0/487317276 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff9700a2fa0 msgr2=0x7ff9700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff9700a2fa0 0x7ff9700a5460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff97802c240 tx=0x7ff97802df90 comp rx=0 tx=0).stop 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 -- 172.21.15.49:0/487317276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff988104d10 msgr2=0x7ff988100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff988104d10 0x7ff988100390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff97c005fd0 tx=0x7ff97c00dde0 comp rx=0 tx=0).stop 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 -- 172.21.15.49:0/487317276 shutdown_connections 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff9700a2fa0 0x7ff9700a5460 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff988107640 0x7ff9881008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff988104d10 0x7ff988100390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 --2- 172.21.15.49:0/487317276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff9881023e0 0x7ff988101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 -- 172.21.15.49:0/487317276 >> 172.21.15.49:0/487317276 conn(0x7ff9880fbdf0 msgr2=0x7ff988102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 -- 172.21.15.49:0/487317276 shutdown_connections 2024-03-26T17:59:22.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:22.292+0000 7ff98edc4640 1 -- 172.21.15.49:0/487317276 wait complete. 2024-03-26T17:59:22.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:22 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/487317276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:23.107 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:22 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/487317276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:23.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:22 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/487317276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:23.298 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 17:59:23 smithi115 podman[65833]: 2024-03-26 17:59:23.292326259 +0000 UTC m=+0.364540588 container start 82c9270a6458d440ceb72572cb94cfc82bbd2149ac69bfbdfa0670947c013b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-26T17:59:23.298 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 17:59:23 smithi115 ceph-mds[65852]: -- 172.21.15.115:0/2418788821 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c73dc75a40 con 0x55c73dc48c00 2024-03-26T17:59:23.738 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 17:59:23 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-e[65848]: starting mds.e at 2024-03-26T17:59:23.738 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 17:59:23 smithi115 bash[65833]: 82c9270a6458d440ceb72572cb94cfc82bbd2149ac69bfbdfa0670947c013b65 2024-03-26T17:59:23.738 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 17:59:23 smithi115 systemd[1]: Started Ceph mds.e for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:24.286 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:24.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.424+0000 7f5db0a7e640 1 -- 172.21.15.49:0/193627880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dac100860 msgr2=0x7f5dac100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.424+0000 7f5db0a7e640 1 --2- 172.21.15.49:0/193627880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dac100860 0x7f5dac100c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009900 tx=0x7f5d9c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.424+0000 7f5db0a7e640 1 -- 172.21.15.49:0/193627880 shutdown_connections 2024-03-26T17:59:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.424+0000 7f5db0a7e640 1 --2- 172.21.15.49:0/193627880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dac101a50 0x7f5dac101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.424+0000 7f5db0a7e640 1 --2- 172.21.15.49:0/193627880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5dac102410 0x7f5dac108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.424+0000 7f5db0a7e640 1 --2- 172.21.15.49:0/193627880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dac100860 0x7f5dac100c80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.424+0000 7f5db0a7e640 1 -- 172.21.15.49:0/193627880 >> 172.21.15.49:0/193627880 conn(0x7f5dac0fbdf0 msgr2=0x7f5dac0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.424+0000 7f5db0a7e640 1 -- 172.21.15.49:0/193627880 shutdown_connections 2024-03-26T17:59:24.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.424+0000 7f5db0a7e640 1 -- 172.21.15.49:0/193627880 wait complete. 2024-03-26T17:59:24.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.425+0000 7f5db0a7e640 1 Processor -- start 2024-03-26T17:59:24.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.425+0000 7f5db0a7e640 1 -- start start 2024-03-26T17:59:24.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5db0a7e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dac100860 0x7f5dac06aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5db0a7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dac101a50 0x7f5dac19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5db0a7e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5dac102410 0x7f5dac1a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5db0a7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac10c460 con 0x7f5dac101a50 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5db0a7e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac10c2e0 con 0x7f5dac102410 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5db0a7e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac10c5e0 con 0x7f5dac100860 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da1d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dac101a50 0x7f5dac19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da1d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dac101a50 0x7f5dac19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55142/0 (socket says 172.21.15.49:55142) 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da1d9b640 1 -- 172.21.15.49:0/2452706767 learned_addr learned my addr 172.21.15.49:0/2452706767 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5daa59c640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dac100860 0x7f5dac06aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5daad9d640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5dac102410 0x7f5dac1a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da1d9b640 1 -- 172.21.15.49:0/2452706767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dac100860 msgr2=0x7f5dac06aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da1d9b640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dac100860 0x7f5dac06aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da1d9b640 1 -- 172.21.15.49:0/2452706767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5dac102410 msgr2=0x7f5dac1a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da1d9b640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5dac102410 0x7f5dac1a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da1d9b640 1 -- 172.21.15.49:0/2452706767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac1a1aa0 con 0x7f5dac101a50 2024-03-26T17:59:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da1d9b640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dac101a50 0x7f5dac19ed60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e9f0 tx=0x7f5d9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da3fff640 1 -- 172.21.15.49:0/2452706767 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400cd80 con 0x7f5dac101a50 2024-03-26T17:59:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5db0a7e640 1 -- 172.21.15.49:0/2452706767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac1a1d30 con 0x7f5dac101a50 2024-03-26T17:59:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da3fff640 1 -- 172.21.15.49:0/2452706767 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d94005500 con 0x7f5dac101a50 2024-03-26T17:59:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.426+0000 7f5da3fff640 1 -- 172.21.15.49:0/2452706767 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94010690 con 0x7f5dac101a50 2024-03-26T17:59:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.427+0000 7f5db0a7e640 1 -- 172.21.15.49:0/2452706767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac1a2220 con 0x7f5dac101a50 2024-03-26T17:59:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.427+0000 7f5db0a7e640 1 -- 172.21.15.49:0/2452706767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dac107bf0 con 0x7f5dac101a50 2024-03-26T17:59:24.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.429+0000 7f5da3fff640 1 -- 172.21.15.49:0/2452706767 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f5d940046c0 con 0x7f5dac101a50 2024-03-26T17:59:24.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.429+0000 7f5da3fff640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d980a2f50 0x7f5d980a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.429+0000 7f5da3fff640 1 -- 172.21.15.49:0/2452706767 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5d940cf470 con 0x7f5dac101a50 2024-03-26T17:59:24.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.429+0000 7f5daa59c640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d980a2f50 0x7f5d980a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.430+0000 7f5daa59c640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d980a2f50 0x7f5d980a5410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c02c7b0 tx=0x7f5d9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.434+0000 7f5da3fff640 1 -- 172.21.15.49:0/2452706767 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5d9409c740 con 0x7f5dac101a50 2024-03-26T17:59:24.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.656+0000 7f5db0a7e640 1 -- 172.21.15.49:0/2452706767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dac02d240 con 0x7f5dac101a50 2024-03-26T17:59:24.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.657+0000 7f5da3fff640 1 -- 172.21.15.49:0/2452706767 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7f5d9409c040 con 0x7f5dac101a50 2024-03-26T17:59:24.640 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:24.640 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-26T17:59:24.640 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 6 2024-03-26T17:59:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.658+0000 7f5db0a7e640 1 -- 172.21.15.49:0/2452706767 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d980a2f50 msgr2=0x7f5d980a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.658+0000 7f5db0a7e640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d980a2f50 0x7f5d980a5410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c02c7b0 tx=0x7f5d9c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.658+0000 7f5db0a7e640 1 -- 172.21.15.49:0/2452706767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dac101a50 msgr2=0x7f5dac19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.658+0000 7f5db0a7e640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dac101a50 0x7f5dac19ed60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e9f0 tx=0x7f5d9400edf0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.659+0000 7f5db0a7e640 1 -- 172.21.15.49:0/2452706767 shutdown_connections 2024-03-26T17:59:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.659+0000 7f5db0a7e640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d980a2f50 0x7f5d980a5410 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.659+0000 7f5db0a7e640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5dac102410 0x7f5dac1a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.659+0000 7f5db0a7e640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5dac101a50 0x7f5dac19ed60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.659+0000 7f5db0a7e640 1 --2- 172.21.15.49:0/2452706767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dac100860 0x7f5dac06aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.659+0000 7f5db0a7e640 1 -- 172.21.15.49:0/2452706767 >> 172.21.15.49:0/2452706767 conn(0x7f5dac0fbdf0 msgr2=0x7f5dac104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.659+0000 7f5db0a7e640 1 -- 172.21.15.49:0/2452706767 shutdown_connections 2024-03-26T17:59:24.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:24.659+0000 7f5db0a7e640 1 -- 172.21.15.49:0/2452706767 wait complete. 2024-03-26T17:59:24.651 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: Deploying daemon mds.f on smithi171 2024-03-26T17:59:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6834/4085026954,v1:172.21.15.115:6835/4085026954] up:boot 2024-03-26T17:59:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: fsmap 5 up:standby 2024-03-26T17:59:24.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:24 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:59:24.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:24.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:24.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:24.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:24.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:24.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:24.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:24.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: Deploying daemon mds.f on smithi171 2024-03-26T17:59:24.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6834/4085026954,v1:172.21.15.115:6835/4085026954] up:boot 2024-03-26T17:59:24.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: fsmap 5 up:standby 2024-03-26T17:59:24.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:24 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:59:24.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:24.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:24.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:24.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:24.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:24.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:24.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:24.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: Deploying daemon mds.f on smithi171 2024-03-26T17:59:24.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6834/4085026954,v1:172.21.15.115:6835/4085026954] up:boot 2024-03-26T17:59:24.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: fsmap 5 up:standby 2024-03-26T17:59:24.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:24 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:59:25.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:25 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2452706767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:25.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:25 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2452706767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:25.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:25 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2452706767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:25.789 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 17:59:25 smithi171 systemd[1]: Starting Ceph mds.f for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:59:26.040 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 17:59:25 smithi171 podman[66483]: 2024-03-26 17:59:25.708647166 +0000 UTC m=+0.017245659 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:59:26.040 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 17:59:25 smithi171 podman[66483]: 2024-03-26 17:59:25.820293194 +0000 UTC m=+0.128891668 container create 3e7821f4b5e0c942f59d5a09b3db2fc0b13b81303ce0e09c1ebf843891683835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-26T17:59:26.040 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 17:59:26 smithi171 podman[66483]: 2024-03-26 17:59:26.057346896 +0000 UTC m=+0.365945380 container init 3e7821f4b5e0c942f59d5a09b3db2fc0b13b81303ce0e09c1ebf843891683835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-26T17:59:26.357 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 17:59:26 smithi171 podman[66483]: 2024-03-26 17:59:26.060034717 +0000 UTC m=+0.368633206 container start 3e7821f4b5e0c942f59d5a09b3db2fc0b13b81303ce0e09c1ebf843891683835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-26T17:59:26.357 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 17:59:26 smithi171 ceph-mds[66502]: -- 172.21.15.171:0/4170157587 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e6f7e7a40 con 0x556e6f7bac00 2024-03-26T17:59:26.357 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 17:59:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f[66498]: starting mds.f at 2024-03-26T17:59:26.357 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 17:59:26 smithi171 bash[66483]: 3e7821f4b5e0c942f59d5a09b3db2fc0b13b81303ce0e09c1ebf843891683835 2024-03-26T17:59:26.357 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 17:59:26 smithi171 systemd[1]: Started Ceph mds.f for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:26.654 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:26.687 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:26 smithi049 ceph-mon[32331]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:26.687 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:26.687 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:26.687 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:26.687 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:26.687 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:26.687 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:26 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:26.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:26 smithi115 ceph-mon[36174]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:26.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:26.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:26.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:26.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:26.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:26.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:26 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.793+0000 7fdabec0b640 1 -- 172.21.15.49:0/2591376614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdab81023e0 msgr2=0x7fdab81047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.793+0000 7fdabec0b640 1 --2- 172.21.15.49:0/2591376614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdab81023e0 0x7fdab81047d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009900 tx=0x7fdaa802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.793+0000 7fdabec0b640 1 -- 172.21.15.49:0/2591376614 shutdown_connections 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.793+0000 7fdabec0b640 1 --2- 172.21.15.49:0/2591376614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdab8107640 0x7fdab8109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.793+0000 7fdabec0b640 1 --2- 172.21.15.49:0/2591376614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdab8104d10 0x7fdab8107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.793+0000 7fdabec0b640 1 --2- 172.21.15.49:0/2591376614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdab81023e0 0x7fdab81047d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.793+0000 7fdabec0b640 1 -- 172.21.15.49:0/2591376614 >> 172.21.15.49:0/2591376614 conn(0x7fdab80fbdf0 msgr2=0x7fdab80fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.793+0000 7fdabec0b640 1 -- 172.21.15.49:0/2591376614 shutdown_connections 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.793+0000 7fdabec0b640 1 -- 172.21.15.49:0/2591376614 wait complete. 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.793+0000 7fdabec0b640 1 Processor -- start 2024-03-26T17:59:26.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdabec0b640 1 -- start start 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdabec0b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdab81023e0 0x7fdab8197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdabec0b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdab8104d10 0x7fdab8198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdabec0b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdab8107640 0x7fdab819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdabec0b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdab810d570 con 0x7fdab8104d10 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdabec0b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdab810d3f0 con 0x7fdab81023e0 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdabec0b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdab810d6f0 con 0x7fdab8107640 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdab7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdab8104d10 0x7fdab8198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdab7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdab8104d10 0x7fdab8198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55154/0 (socket says 172.21.15.49:55154) 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdab7fff640 1 -- 172.21.15.49:0/3852781155 learned_addr learned my addr 172.21.15.49:0/3852781155 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdabc9a7640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdab81023e0 0x7fdab8197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdabd1a8640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdab8107640 0x7fdab819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdab7fff640 1 -- 172.21.15.49:0/3852781155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdab8107640 msgr2=0x7fdab819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdab7fff640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdab8107640 0x7fdab819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdab7fff640 1 -- 172.21.15.49:0/3852781155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdab81023e0 msgr2=0x7fdab8197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdab7fff640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdab81023e0 0x7fdab8197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdab7fff640 1 -- 172.21.15.49:0/3852781155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab819f840 con 0x7fdab8104d10 2024-03-26T17:59:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdab7fff640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdab8104d10 0x7fdab8198170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00d770 tx=0x7fdaac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.794+0000 7fdab5ffb640 1 -- 172.21.15.49:0/3852781155 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac017070 con 0x7fdab8104d10 2024-03-26T17:59:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.795+0000 7fdabec0b640 1 -- 172.21.15.49:0/3852781155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab819fb30 con 0x7fdab8104d10 2024-03-26T17:59:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.795+0000 7fdab5ffb640 1 -- 172.21.15.49:0/3852781155 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaac009c30 con 0x7fdab8104d10 2024-03-26T17:59:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.795+0000 7fdab5ffb640 1 -- 172.21.15.49:0/3852781155 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac00f620 con 0x7fdab8104d10 2024-03-26T17:59:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.795+0000 7fdabec0b640 1 -- 172.21.15.49:0/3852781155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab81a0080 con 0x7fdab8104d10 2024-03-26T17:59:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.796+0000 7fdabec0b640 1 -- 172.21.15.49:0/3852781155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab80fd8e0 con 0x7fdab8104d10 2024-03-26T17:59:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.797+0000 7fdab5ffb640 1 -- 172.21.15.49:0/3852781155 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fdaac01a070 con 0x7fdab8104d10 2024-03-26T17:59:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.798+0000 7fdab5ffb640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdaa00a2f20 0x7fdaa00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.798+0000 7fdab5ffb640 1 -- 172.21.15.49:0/3852781155 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdaac0cea20 con 0x7fdab8104d10 2024-03-26T17:59:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.798+0000 7fdabc9a7640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdaa00a2f20 0x7fdaa00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.799+0000 7fdabc9a7640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdaa00a2f20 0x7fdaa00a53e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdab8199100 tx=0x7fdaa8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:26.803+0000 7fdab5ffb640 1 -- 172.21.15.49:0/3852781155 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdaac009de0 con 0x7fdab8104d10 2024-03-26T17:59:26.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:26 smithi171 ceph-mon[36033]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:26.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:26.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:26.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:26.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:26.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:26.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:26 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:27.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.025+0000 7fdabec0b640 1 -- 172.21.15.49:0/3852781155 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdab802d240 con 0x7fdab8104d10 2024-03-26T17:59:27.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.026+0000 7fdab5ffb640 1 -- 172.21.15.49:0/3852781155 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7fdaac09b6a0 con 0x7fdab8104d10 2024-03-26T17:59:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:27.009 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-26T17:59:27.009 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.027+0000 7fdabec0b640 1 -- 172.21.15.49:0/3852781155 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdaa00a2f20 msgr2=0x7fdaa00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.027+0000 7fdabec0b640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdaa00a2f20 0x7fdaa00a53e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdab8199100 tx=0x7fdaa8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.027+0000 7fdabec0b640 1 -- 172.21.15.49:0/3852781155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdab8104d10 msgr2=0x7fdab8198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.028+0000 7fdabec0b640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdab8104d10 0x7fdab8198170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00d770 tx=0x7fdaac00dc30 comp rx=0 tx=0).stop 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.028+0000 7fdabec0b640 1 -- 172.21.15.49:0/3852781155 shutdown_connections 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.028+0000 7fdabec0b640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fdaa00a2f20 0x7fdaa00a53e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.028+0000 7fdabec0b640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdab8107640 0x7fdab819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.028+0000 7fdabec0b640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdab8104d10 0x7fdab8198170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.028+0000 7fdabec0b640 1 --2- 172.21.15.49:0/3852781155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdab81023e0 0x7fdab8197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.028+0000 7fdabec0b640 1 -- 172.21.15.49:0/3852781155 >> 172.21.15.49:0/3852781155 conn(0x7fdab80fbdf0 msgr2=0x7fdab8102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:27.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.028+0000 7fdabec0b640 1 -- 172.21.15.49:0/3852781155 shutdown_connections 2024-03-26T17:59:27.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:27.028+0000 7fdabec0b640 1 -- 172.21.15.49:0/3852781155 wait complete. 2024-03-26T17:59:27.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:27 smithi171 ceph-mon[36033]: Deploying daemon mds.c on smithi171 2024-03-26T17:59:27.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:27 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.171:6834/3854985421,v1:172.21.15.171:6835/3854985421] up:boot 2024-03-26T17:59:27.705 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:27 smithi171 ceph-mon[36033]: fsmap 6 up:standby 2024-03-26T17:59:27.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:27 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T17:59:27.706 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:27 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3852781155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:27.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:27 smithi115 ceph-mon[36174]: Deploying daemon mds.c on smithi171 2024-03-26T17:59:27.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:27 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.171:6834/3854985421,v1:172.21.15.171:6835/3854985421] up:boot 2024-03-26T17:59:27.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:27 smithi115 ceph-mon[36174]: fsmap 6 up:standby 2024-03-26T17:59:27.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:27 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T17:59:27.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:27 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3852781155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:27.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:27 smithi049 ceph-mon[32331]: Deploying daemon mds.c on smithi171 2024-03-26T17:59:27.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:27 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.171:6834/3854985421,v1:172.21.15.171:6835/3854985421] up:boot 2024-03-26T17:59:27.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:27 smithi049 ceph-mon[32331]: fsmap 6 up:standby 2024-03-26T17:59:27.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:27 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T17:59:27.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:27 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3852781155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:28.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:28 smithi171 ceph-mon[36033]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:28.624 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 17:59:28 smithi171 systemd[1]: Starting Ceph mds.c for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:59:28.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:28 smithi115 ceph-mon[36174]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:28.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:28 smithi049 ceph-mon[32331]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:28.936 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 17:59:28 smithi171 podman[66789]: 2024-03-26 17:59:28.591364987 +0000 UTC m=+0.018070386 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:59:28.936 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 17:59:28 smithi171 podman[66789]: 2024-03-26 17:59:28.709526434 +0000 UTC m=+0.136231836 container create 259c053f8c6facf7da4f0a8dd0b944c45cb1aa7d10f202420315d3f0787cd7cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-26T17:59:29.022 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:29.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.163+0000 7fc6d56b2640 1 -- 172.21.15.49:0/970021757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d0104cf0 msgr2=0x7fc6d01070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:29.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.163+0000 7fc6d56b2640 1 --2- 172.21.15.49:0/970021757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d0104cf0 0x7fc6d01070e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009900 tx=0x7fc6c402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.163+0000 7fc6d56b2640 1 -- 172.21.15.49:0/970021757 shutdown_connections 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.163+0000 7fc6d56b2640 1 --2- 172.21.15.49:0/970021757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6d0107620 0x7fc6d0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.163+0000 7fc6d56b2640 1 --2- 172.21.15.49:0/970021757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d0104cf0 0x7fc6d01070e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.163+0000 7fc6d56b2640 1 --2- 172.21.15.49:0/970021757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d01023c0 0x7fc6d01047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.163+0000 7fc6d56b2640 1 -- 172.21.15.49:0/970021757 >> 172.21.15.49:0/970021757 conn(0x7fc6d00fbd50 msgr2=0x7fc6d00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.163+0000 7fc6d56b2640 1 -- 172.21.15.49:0/970021757 shutdown_connections 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.163+0000 7fc6d56b2640 1 -- 172.21.15.49:0/970021757 wait complete. 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6d56b2640 1 Processor -- start 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6d56b2640 1 -- start start 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6d56b2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d01023c0 0x7fc6d0197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:29.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6d56b2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6d0104cf0 0x7fc6d0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6d56b2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d0107620 0x7fc6d019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6d56b2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d010d530 con 0x7fc6d01023c0 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6d56b2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d010d3b0 con 0x7fc6d0107620 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6d56b2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d010d6b0 con 0x7fc6d0104cf0 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6ceffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d01023c0 0x7fc6d0197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6ceffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d01023c0 0x7fc6d0197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55998/0 (socket says 172.21.15.49:55998) 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6ceffd640 1 -- 172.21.15.49:0/3175208996 learned_addr learned my addr 172.21.15.49:0/3175208996 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6cf7fe640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d0107620 0x7fc6d019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.164+0000 7fc6ce7fc640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6d0104cf0 0x7fc6d0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6ceffd640 1 -- 172.21.15.49:0/3175208996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6d0104cf0 msgr2=0x7fc6d0198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:29.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6ceffd640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6d0104cf0 0x7fc6d0198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6ceffd640 1 -- 172.21.15.49:0/3175208996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d0107620 msgr2=0x7fc6d019f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:29.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6ceffd640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d0107620 0x7fc6d019f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6ceffd640 1 -- 172.21.15.49:0/3175208996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d019f860 con 0x7fc6d01023c0 2024-03-26T17:59:29.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6ceffd640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d01023c0 0x7fc6d0197c50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000e9f0 tx=0x7fc6c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:29.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6b7fff640 1 -- 172.21.15.49:0/3175208996 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000cd80 con 0x7fc6d01023c0 2024-03-26T17:59:29.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6b7fff640 1 -- 172.21.15.49:0/3175208996 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6c0010910 con 0x7fc6d01023c0 2024-03-26T17:59:29.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6d56b2640 1 -- 172.21.15.49:0/3175208996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d019fb50 con 0x7fc6d01023c0 2024-03-26T17:59:29.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6b7fff640 1 -- 172.21.15.49:0/3175208996 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0018990 con 0x7fc6d01023c0 2024-03-26T17:59:29.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.165+0000 7fc6d56b2640 1 -- 172.21.15.49:0/3175208996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d01a00a0 con 0x7fc6d01023c0 2024-03-26T17:59:29.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.166+0000 7fc6d56b2640 1 -- 172.21.15.49:0/3175208996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d00fd840 con 0x7fc6d01023c0 2024-03-26T17:59:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.167+0000 7fc6b7fff640 1 -- 172.21.15.49:0/3175208996 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fc6c0010ac0 con 0x7fc6d01023c0 2024-03-26T17:59:29.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.168+0000 7fc6b7fff640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6b80a2f20 0x7fc6b80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:29.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.168+0000 7fc6ce7fc640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6b80a2f20 0x7fc6b80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:29.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.168+0000 7fc6b7fff640 1 -- 172.21.15.49:0/3175208996 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc6c00cf6f0 con 0x7fc6d01023c0 2024-03-26T17:59:29.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.168+0000 7fc6ce7fc640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6b80a2f20 0x7fc6b80a53e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4000c00 tx=0x7fc6c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:29.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.172+0000 7fc6b7fff640 1 -- 172.21.15.49:0/3175208996 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc6c009c370 con 0x7fc6d01023c0 2024-03-26T17:59:29.357 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 17:59:28 smithi171 podman[66789]: 2024-03-26 17:59:28.954076792 +0000 UTC m=+0.380782185 container init 259c053f8c6facf7da4f0a8dd0b944c45cb1aa7d10f202420315d3f0787cd7cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T17:59:29.357 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 17:59:28 smithi171 podman[66789]: 2024-03-26 17:59:28.956738487 +0000 UTC m=+0.383443872 container start 259c053f8c6facf7da4f0a8dd0b944c45cb1aa7d10f202420315d3f0787cd7cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-26T17:59:29.357 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 17:59:28 smithi171 ceph-mds[66808]: -- 172.21.15.171:0/3785317058 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5623f9755a40 con 0x5623f9729000 2024-03-26T17:59:29.357 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 17:59:28 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c[66804]: starting mds.c at 2024-03-26T17:59:29.358 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 17:59:29 smithi171 bash[66789]: 259c053f8c6facf7da4f0a8dd0b944c45cb1aa7d10f202420315d3f0787cd7cf 2024-03-26T17:59:29.358 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 17:59:29 smithi171 systemd[1]: Started Ceph mds.c for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.394+0000 7fc6d56b2640 1 -- 172.21.15.49:0/3175208996 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6d002d240 con 0x7fc6d01023c0 2024-03-26T17:59:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.394+0000 7fc6b7fff640 1 -- 172.21.15.49:0/3175208996 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7fc6c00a0410 con 0x7fc6d01023c0 2024-03-26T17:59:29.378 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:29.378 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-26T17:59:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 7 2024-03-26T17:59:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 -- 172.21.15.49:0/3175208996 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6b80a2f20 msgr2=0x7fc6b80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6b80a2f20 0x7fc6b80a53e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4000c00 tx=0x7fc6c4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 -- 172.21.15.49:0/3175208996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d01023c0 msgr2=0x7fc6d0197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d01023c0 0x7fc6d0197c50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000e9f0 tx=0x7fc6c000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 -- 172.21.15.49:0/3175208996 shutdown_connections 2024-03-26T17:59:29.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6b80a2f20 0x7fc6b80a53e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6d0104cf0 0x7fc6d0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d0107620 0x7fc6d019f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 --2- 172.21.15.49:0/3175208996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6d01023c0 0x7fc6d0197c50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:29.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 -- 172.21.15.49:0/3175208996 >> 172.21.15.49:0/3175208996 conn(0x7fc6d00fbd50 msgr2=0x7fc6d0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:29.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 -- 172.21.15.49:0/3175208996 shutdown_connections 2024-03-26T17:59:29.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:29.396+0000 7fc6d56b2640 1 -- 172.21.15.49:0/3175208996 wait complete. 2024-03-26T17:59:30.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: Deploying daemon mds.h on smithi115 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3175208996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.171:6836/2428288879,v1:172.21.15.171:6837/2428288879] up:boot 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: fsmap 7 up:standby 2024-03-26T17:59:30.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:30 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T17:59:30.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:30.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:30.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:30.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:30.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:30.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:30.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:30.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: Deploying daemon mds.h on smithi115 2024-03-26T17:59:30.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3175208996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:30.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.171:6836/2428288879,v1:172.21.15.171:6837/2428288879] up:boot 2024-03-26T17:59:30.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: fsmap 7 up:standby 2024-03-26T17:59:30.358 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:30 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T17:59:30.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:30.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:30.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:30.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:30.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:30.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:30.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:30.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: Deploying daemon mds.h on smithi115 2024-03-26T17:59:30.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3175208996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:30.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.171:6836/2428288879,v1:172.21.15.171:6837/2428288879] up:boot 2024-03-26T17:59:30.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: fsmap 7 up:standby 2024-03-26T17:59:30.504 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:30 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T17:59:31.392 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:31.488 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 17:59:31 smithi115 systemd[1]: Starting Ceph mds.h for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:59:31.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.529+0000 7f8812b5b640 1 -- 172.21.15.49:0/1740511391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f880c102410 msgr2=0x7f880c108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:31.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.529+0000 7f8812b5b640 1 --2- 172.21.15.49:0/1740511391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f880c102410 0x7f880c108ca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ca0 tx=0x7f880402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.529+0000 7f8812b5b640 1 -- 172.21.15.49:0/1740511391 shutdown_connections 2024-03-26T17:59:31.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.529+0000 7f8812b5b640 1 --2- 172.21.15.49:0/1740511391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f880c102410 0x7f880c108ca0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.529+0000 7f8812b5b640 1 --2- 172.21.15.49:0/1740511391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f880c101a50 0x7f880c101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.529+0000 7f8812b5b640 1 --2- 172.21.15.49:0/1740511391 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f880c100860 0x7f880c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.529+0000 7f8812b5b640 1 -- 172.21.15.49:0/1740511391 >> 172.21.15.49:0/1740511391 conn(0x7f880c0fbdf0 msgr2=0x7f880c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:31.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.529+0000 7f8812b5b640 1 -- 172.21.15.49:0/1740511391 shutdown_connections 2024-03-26T17:59:31.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.529+0000 7f8812b5b640 1 -- 172.21.15.49:0/1740511391 wait complete. 2024-03-26T17:59:31.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f8812b5b640 1 Processor -- start 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f8812b5b640 1 -- start start 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f8812b5b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f880c100860 0x7f880c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f8812b5b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f880c101a50 0x7f880c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f8812b5b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f880c102410 0x7f880c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f8812b5b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10d590 con 0x7f880c102410 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f8812b5b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10d410 con 0x7f880c100860 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f8812b5b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10d710 con 0x7f880c101a50 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f88110f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f880c102410 0x7f880c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f88110f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f880c102410 0x7f880c19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56008/0 (socket says 172.21.15.49:56008) 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f88110f8640 1 -- 172.21.15.49:0/3441672938 learned_addr learned my addr 172.21.15.49:0/3441672938 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f88108f7640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f880c100860 0x7f880c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f880bfff640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f880c101a50 0x7f880c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f88110f8640 1 -- 172.21.15.49:0/3441672938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f880c101a50 msgr2=0x7f880c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:31.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f88110f8640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f880c101a50 0x7f880c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f88110f8640 1 -- 172.21.15.49:0/3441672938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f880c100860 msgr2=0x7f880c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:31.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f88110f8640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f880c100860 0x7f880c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.530+0000 7f88110f8640 1 -- 172.21.15.49:0/3441672938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c19f7a0 con 0x7f880c102410 2024-03-26T17:59:31.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.531+0000 7f88110f8640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f880c102410 0x7f880c19f130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f880402c630 tx=0x7f8804032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:31.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.531+0000 7f8809ffb640 1 -- 172.21.15.49:0/3441672938 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804041020 con 0x7f880c102410 2024-03-26T17:59:31.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.531+0000 7f8809ffb640 1 -- 172.21.15.49:0/3441672938 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804034cd0 con 0x7f880c102410 2024-03-26T17:59:31.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.531+0000 7f8812b5b640 1 -- 172.21.15.49:0/3441672938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c19fa30 con 0x7f880c102410 2024-03-26T17:59:31.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.531+0000 7f8809ffb640 1 -- 172.21.15.49:0/3441672938 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804039040 con 0x7f880c102410 2024-03-26T17:59:31.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.531+0000 7f8812b5b640 1 -- 172.21.15.49:0/3441672938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c19ff20 con 0x7f880c102410 2024-03-26T17:59:31.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.532+0000 7f8812b5b640 1 -- 172.21.15.49:0/3441672938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c107b80 con 0x7f880c102410 2024-03-26T17:59:31.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.533+0000 7f8809ffb640 1 -- 172.21.15.49:0/3441672938 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f8804044020 con 0x7f880c102410 2024-03-26T17:59:31.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.534+0000 7f8809ffb640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f87fc0a2f20 0x7f87fc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:31.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.534+0000 7f88108f7640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f87fc0a2f20 0x7f87fc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:31.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.534+0000 7f8809ffb640 1 -- 172.21.15.49:0/3441672938 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f88040f0ef0 con 0x7f880c102410 2024-03-26T17:59:31.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.534+0000 7f88108f7640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f87fc0a2f20 0x7f87fc0a53e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f880c1018b0 tx=0x7f8800005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:31.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.538+0000 7f8809ffb640 1 -- 172.21.15.49:0/3441672938 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f880404b020 con 0x7f880c102410 2024-03-26T17:59:31.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.761+0000 7f8812b5b640 1 -- 172.21.15.49:0/3441672938 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f880c02d620 con 0x7f880c102410 2024-03-26T17:59:31.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.761+0000 7f8809ffb640 1 -- 172.21.15.49:0/3441672938 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f880c02d620 con 0x7f880c102410 2024-03-26T17:59:31.745 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:31.746 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-26T17:59:31.746 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2024-03-26T17:59:31.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.763+0000 7f8812b5b640 1 -- 172.21.15.49:0/3441672938 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f87fc0a2f20 msgr2=0x7f87fc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:31.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.763+0000 7f8812b5b640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f87fc0a2f20 0x7f87fc0a53e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f880c1018b0 tx=0x7f8800005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:31.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.763+0000 7f8812b5b640 1 -- 172.21.15.49:0/3441672938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f880c102410 msgr2=0x7f880c19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:31.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.763+0000 7f8812b5b640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f880c102410 0x7f880c19f130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f880402c630 tx=0x7f8804032ea0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.763+0000 7f8812b5b640 1 -- 172.21.15.49:0/3441672938 shutdown_connections 2024-03-26T17:59:31.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.763+0000 7f8812b5b640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f87fc0a2f20 0x7f87fc0a53e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.763+0000 7f8812b5b640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f880c101a50 0x7f880c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.763+0000 7f8812b5b640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f880c102410 0x7f880c19f130 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.763+0000 7f8812b5b640 1 --2- 172.21.15.49:0/3441672938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f880c100860 0x7f880c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:31.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.763+0000 7f8812b5b640 1 -- 172.21.15.49:0/3441672938 >> 172.21.15.49:0/3441672938 conn(0x7f880c0fbdf0 msgr2=0x7f880c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:31.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.764+0000 7f8812b5b640 1 -- 172.21.15.49:0/3441672938 shutdown_connections 2024-03-26T17:59:31.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:31.764+0000 7f8812b5b640 1 -- 172.21.15.49:0/3441672938 wait complete. 2024-03-26T17:59:31.826 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 17:59:31 smithi115 podman[66143]: 2024-03-26 17:59:31.4948444 +0000 UTC m=+0.017749342 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:59:31.826 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 17:59:31 smithi115 podman[66143]: 2024-03-26 17:59:31.599396271 +0000 UTC m=+0.122301205 container create 2420a64b3c8eeecc8826d7beb2cba6406646addf44f5ee05fdcbb8eff29dbc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS) 2024-03-26T17:59:31.827 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 17:59:31 smithi115 podman[66143]: 2024-03-26 17:59:31.818867509 +0000 UTC m=+0.341772443 container init 2420a64b3c8eeecc8826d7beb2cba6406646addf44f5ee05fdcbb8eff29dbc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T17:59:31.827 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 17:59:31 smithi115 podman[66143]: 2024-03-26 17:59:31.821513502 +0000 UTC m=+0.344418429 container start 2420a64b3c8eeecc8826d7beb2cba6406646addf44f5ee05fdcbb8eff29dbc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T17:59:32.090 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 17:59:31 smithi115 ceph-mds[66161]: -- 172.21.15.115:0/907912876 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d87db9da40 con 0x55d87db71000 2024-03-26T17:59:32.091 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 17:59:31 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h[66157]: starting mds.h at 2024-03-26T17:59:32.091 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 17:59:31 smithi115 bash[66143]: 2420a64b3c8eeecc8826d7beb2cba6406646addf44f5ee05fdcbb8eff29dbc8d 2024-03-26T17:59:32.091 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 17:59:31 smithi115 systemd[1]: Started Ceph mds.h for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:32.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:32 smithi171 ceph-mon[36033]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:32.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:32 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3441672938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:32.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:32 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:32.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:32 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:32.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:32 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:32.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:32 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:32.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:32 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:32.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:32 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:32.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:32 smithi115 ceph-mon[36174]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:32.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:32 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3441672938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:32.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:32 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:32.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:32 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:32.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:32 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:32.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:32 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:32.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:32 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:32.489 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:32 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:32.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:32 smithi049 ceph-mon[32331]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:32.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:32 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3441672938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:32.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:32 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:32.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:32 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:32.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:32 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:32.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:32 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:32.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:32 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:32.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:32 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:33.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:33 smithi171 ceph-mon[36033]: Deploying daemon mds.i on smithi171 2024-03-26T17:59:33.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:33 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] up:boot 2024-03-26T17:59:33.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:33 smithi171 ceph-mon[36033]: fsmap 8 up:standby 2024-03-26T17:59:33.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:33 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T17:59:33.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:33 smithi115 ceph-mon[36174]: Deploying daemon mds.i on smithi171 2024-03-26T17:59:33.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:33 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] up:boot 2024-03-26T17:59:33.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:33 smithi115 ceph-mon[36174]: fsmap 8 up:standby 2024-03-26T17:59:33.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:33 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T17:59:33.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:33 smithi049 ceph-mon[32331]: Deploying daemon mds.i on smithi171 2024-03-26T17:59:33.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:33 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] up:boot 2024-03-26T17:59:33.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:33 smithi049 ceph-mon[32331]: fsmap 8 up:standby 2024-03-26T17:59:33.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:33 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T17:59:33.759 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.895+0000 7f9963814640 1 -- 172.21.15.49:0/2004129945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f995c1018e0 msgr2=0x7f995c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.895+0000 7f9963814640 1 --2- 172.21.15.49:0/2004129945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f995c1018e0 0x7f995c101d00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9958009900 tx=0x7f995802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 -- 172.21.15.49:0/2004129945 shutdown_connections 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 --2- 172.21.15.49:0/2004129945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f995c102ad0 0x7f995c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 --2- 172.21.15.49:0/2004129945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f995c103490 0x7f995c109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 --2- 172.21.15.49:0/2004129945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f995c1018e0 0x7f995c101d00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 -- 172.21.15.49:0/2004129945 >> 172.21.15.49:0/2004129945 conn(0x7f995c0fce70 msgr2=0x7f995c0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 -- 172.21.15.49:0/2004129945 shutdown_connections 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 -- 172.21.15.49:0/2004129945 wait complete. 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 Processor -- start 2024-03-26T17:59:33.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 -- start start 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f995c1018e0 0x7f995c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f995c102ad0 0x7f995c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f995c103490 0x7f995c19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f995c10d550 con 0x7f995c103490 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f995c10d3d0 con 0x7f995c102ad0 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.896+0000 7f9963814640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f995c10d6d0 con 0x7f995c1018e0 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99615b0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f995c1018e0 0x7f995c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f9961db1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f995c103490 0x7f995c19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99615b0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f995c1018e0 0x7f995c197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:57668/0 (socket says 172.21.15.49:57668) 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99615b0640 1 -- 172.21.15.49:0/3174752621 learned_addr learned my addr 172.21.15.49:0/3174752621 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f9960daf640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f995c102ad0 0x7f995c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99615b0640 1 -- 172.21.15.49:0/3174752621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f995c102ad0 msgr2=0x7f995c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:33.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99615b0640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f995c102ad0 0x7f995c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:33.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99615b0640 1 -- 172.21.15.49:0/3174752621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f995c103490 msgr2=0x7f995c19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:33.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99615b0640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f995c103490 0x7f995c19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:33.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99615b0640 1 -- 172.21.15.49:0/3174752621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c19f850 con 0x7f995c1018e0 2024-03-26T17:59:33.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f9961db1640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f995c103490 0x7f995c19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:33.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99615b0640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f995c1018e0 0x7f995c197c40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9958000c00 tx=0x7f9958005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:33.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99527fc640 1 -- 172.21.15.49:0/3174752621 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958041070 con 0x7f995c1018e0 2024-03-26T17:59:33.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.897+0000 7f99527fc640 1 -- 172.21.15.49:0/3174752621 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99580044d0 con 0x7f995c1018e0 2024-03-26T17:59:33.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.898+0000 7f99527fc640 1 -- 172.21.15.49:0/3174752621 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958034440 con 0x7f995c1018e0 2024-03-26T17:59:33.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.898+0000 7f9963814640 1 -- 172.21.15.49:0/3174752621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995c19fae0 con 0x7f995c1018e0 2024-03-26T17:59:33.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.898+0000 7f9963814640 1 -- 172.21.15.49:0/3174752621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995c1a0020 con 0x7f995c1018e0 2024-03-26T17:59:33.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.898+0000 7f9963814640 1 -- 172.21.15.49:0/3174752621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995c108ca0 con 0x7f995c1018e0 2024-03-26T17:59:33.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.900+0000 7f99527fc640 1 -- 172.21.15.49:0/3174752621 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f9958044070 con 0x7f995c1018e0 2024-03-26T17:59:33.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.900+0000 7f99527fc640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99480a3020 0x7f99480a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:33.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.901+0000 7f9960daf640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99480a3020 0x7f99480a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:33.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.901+0000 7f99527fc640 1 -- 172.21.15.49:0/3174752621 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f99580f0da0 con 0x7f995c1018e0 2024-03-26T17:59:33.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.901+0000 7f9960daf640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99480a3020 0x7f99480a54e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f994c009b60 tx=0x7f994c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:33.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:33.904+0000 7f99527fc640 1 -- 172.21.15.49:0/3174752621 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f99580bdfd0 con 0x7f995c1018e0 2024-03-26T17:59:34.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.128+0000 7f9963814640 1 -- 172.21.15.49:0/3174752621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f995c02d240 con 0x7f995c1018e0 2024-03-26T17:59:34.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.129+0000 7f99527fc640 1 -- 172.21.15.49:0/3174752621 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f99580bd8d0 con 0x7f995c1018e0 2024-03-26T17:59:34.113 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:34.113 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-26T17:59:34.113 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 9 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 -- 172.21.15.49:0/3174752621 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99480a3020 msgr2=0x7f99480a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99480a3020 0x7f99480a54e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f994c009b60 tx=0x7f994c005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 -- 172.21.15.49:0/3174752621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f995c1018e0 msgr2=0x7f995c197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f995c1018e0 0x7f995c197c40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9958000c00 tx=0x7f9958005d50 comp rx=0 tx=0).stop 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 -- 172.21.15.49:0/3174752621 shutdown_connections 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99480a3020 0x7f99480a54e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f995c103490 0x7f995c19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f995c102ad0 0x7f995c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 --2- 172.21.15.49:0/3174752621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f995c1018e0 0x7f995c197c40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 -- 172.21.15.49:0/3174752621 >> 172.21.15.49:0/3174752621 conn(0x7f995c0fce70 msgr2=0x7f995c105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 -- 172.21.15.49:0/3174752621 shutdown_connections 2024-03-26T17:59:34.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:34.131+0000 7f9963814640 1 -- 172.21.15.49:0/3174752621 wait complete. 2024-03-26T17:59:34.300 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:34 smithi171 ceph-mon[36033]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:34.301 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 17:59:34 smithi171 systemd[1]: Starting Ceph mds.i for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:59:34.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:34 smithi115 ceph-mon[36174]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:34.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:34 smithi049 ceph-mon[32331]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:34.576 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 17:59:34 smithi171 podman[67092]: 2024-03-26 17:59:34.256707747 +0000 UTC m=+0.017911211 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:59:34.576 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 17:59:34 smithi171 podman[67092]: 2024-03-26 17:59:34.374117995 +0000 UTC m=+0.135321471 container create 2f2c017b3885e772e5694488d33a50383ab61fdb0508176c1f1a07045d5f08cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-26T17:59:34.576 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 17:59:34 smithi171 podman[67092]: 2024-03-26 17:59:34.593293167 +0000 UTC m=+0.354496636 container init 2f2c017b3885e772e5694488d33a50383ab61fdb0508176c1f1a07045d5f08cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T17:59:34.857 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 17:59:34 smithi171 podman[67092]: 2024-03-26 17:59:34.596634313 +0000 UTC m=+0.357837782 container start 2f2c017b3885e772e5694488d33a50383ab61fdb0508176c1f1a07045d5f08cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T17:59:34.857 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 17:59:34 smithi171 ceph-mds[67111]: -- 172.21.15.171:0/3289689994 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b6bb9fa40 con 0x559b6bb72c00 2024-03-26T17:59:34.857 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 17:59:34 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i[67107]: starting mds.i at 2024-03-26T17:59:34.857 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 17:59:34 smithi171 bash[67092]: 2f2c017b3885e772e5694488d33a50383ab61fdb0508176c1f1a07045d5f08cb 2024-03-26T17:59:34.857 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 17:59:34 smithi171 systemd[1]: Started Ceph mds.i for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:35.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:35 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3174752621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:35.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:35.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:35.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:35.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:35.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:35.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:35 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:35.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:35 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3174752621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:35.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:35.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:35.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:35.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:35.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:35.404 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:35 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:35.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:35 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3174752621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:35.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:35.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:35.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:35.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:35.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:35.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:35 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:36.127 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:36.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.263+0000 7f4c11532640 1 -- 172.21.15.49:0/3955013296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c0c104d10 msgr2=0x7f4c0c107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:36.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.263+0000 7f4c11532640 1 --2- 172.21.15.49:0/3955013296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c0c104d10 0x7f4c0c107100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009900 tx=0x7f4bf402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.263+0000 7f4c11532640 1 -- 172.21.15.49:0/3955013296 shutdown_connections 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.263+0000 7f4c11532640 1 --2- 172.21.15.49:0/3955013296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c107640 0x7f4c0c109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.263+0000 7f4c11532640 1 --2- 172.21.15.49:0/3955013296 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c0c104d10 0x7f4c0c107100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.263+0000 7f4c11532640 1 --2- 172.21.15.49:0/3955013296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c0c1023e0 0x7f4c0c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.263+0000 7f4c11532640 1 -- 172.21.15.49:0/3955013296 >> 172.21.15.49:0/3955013296 conn(0x7f4c0c0fbdf0 msgr2=0x7f4c0c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.263+0000 7f4c11532640 1 -- 172.21.15.49:0/3955013296 shutdown_connections 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.263+0000 7f4c11532640 1 -- 172.21.15.49:0/3955013296 wait complete. 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c11532640 1 Processor -- start 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c11532640 1 -- start start 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c11532640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c0c1023e0 0x7f4c0c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c11532640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c0c104d10 0x7f4c0c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c11532640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c107640 0x7f4c0c19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:36.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c11532640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c10d560 con 0x7f4c0c1023e0 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c11532640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c10d3e0 con 0x7f4c0c107640 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c11532640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c10d6e0 con 0x7f4c0c104d10 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c0affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c0c1023e0 0x7f4c0c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c0affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c0c1023e0 0x7f4c0c197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:56060/0 (socket says 172.21.15.49:56060) 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.264+0000 7f4c0affd640 1 -- 172.21.15.49:0/3536496131 learned_addr learned my addr 172.21.15.49:0/3536496131 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c0b7fe640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c107640 0x7f4c0c19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c027fc640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c0c104d10 0x7f4c0c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c0affd640 1 -- 172.21.15.49:0/3536496131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c0c104d10 msgr2=0x7f4c0c1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c0affd640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c0c104d10 0x7f4c0c1980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c0affd640 1 -- 172.21.15.49:0/3536496131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c107640 msgr2=0x7f4c0c19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c0affd640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c107640 0x7f4c0c19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c0affd640 1 -- 172.21.15.49:0/3536496131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c19f7c0 con 0x7f4c0c1023e0 2024-03-26T17:59:36.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c0b7fe640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c107640 0x7f4c0c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:36.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c0affd640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c0c1023e0 0x7f4c0c197ba0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00c920 tx=0x7f4bfc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:36.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c08ff9640 1 -- 172.21.15.49:0/3536496131 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc007c70 con 0x7f4c0c1023e0 2024-03-26T17:59:36.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c08ff9640 1 -- 172.21.15.49:0/3536496131 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bfc007e10 con 0x7f4c0c1023e0 2024-03-26T17:59:36.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c11532640 1 -- 172.21.15.49:0/3536496131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c19fab0 con 0x7f4c0c1023e0 2024-03-26T17:59:36.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c08ff9640 1 -- 172.21.15.49:0/3536496131 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc018890 con 0x7f4c0c1023e0 2024-03-26T17:59:36.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.265+0000 7f4c11532640 1 -- 172.21.15.49:0/3536496131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c1a0000 con 0x7f4c0c1023e0 2024-03-26T17:59:36.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.266+0000 7f4c11532640 1 -- 172.21.15.49:0/3536496131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0c0fd8e0 con 0x7f4c0c1023e0 2024-03-26T17:59:36.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.267+0000 7f4c08ff9640 1 -- 172.21.15.49:0/3536496131 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f4bfc018a70 con 0x7f4c0c1023e0 2024-03-26T17:59:36.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.268+0000 7f4c08ff9640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4bf80a2f20 0x7f4bf80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:36.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.268+0000 7f4c027fc640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4bf80a2f20 0x7f4bf80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:36.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.268+0000 7f4c08ff9640 1 -- 172.21.15.49:0/3536496131 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4bfc0cf150 con 0x7f4c0c1023e0 2024-03-26T17:59:36.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.268+0000 7f4c027fc640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4bf80a2f20 0x7f4bf80a53e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4bf402c8e0 tx=0x7f4bf4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:36.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.272+0000 7f4c08ff9640 1 -- 172.21.15.49:0/3536496131 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4bfc014030 con 0x7f4c0c1023e0 2024-03-26T17:59:36.447 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:36 smithi049 ceph-mon[32331]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:36.447 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:36 smithi049 ceph-mon[32331]: Deploying daemon mds.g on smithi049 2024-03-26T17:59:36.447 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:36 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] up:boot 2024-03-26T17:59:36.447 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:36 smithi049 ceph-mon[32331]: fsmap 9 up:standby 2024-03-26T17:59:36.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:36 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T17:59:36.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.499+0000 7f4c11532640 1 -- 172.21.15.49:0/3536496131 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c0c02d4d0 con 0x7f4c0c1023e0 2024-03-26T17:59:36.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.499+0000 7f4c08ff9640 1 -- 172.21.15.49:0/3536496131 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f4bfc00de20 con 0x7f4c0c1023e0 2024-03-26T17:59:36.483 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:36.483 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-26T17:59:36.483 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 10 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.501+0000 7f4c11532640 1 -- 172.21.15.49:0/3536496131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4bf80a2f20 msgr2=0x7f4bf80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.501+0000 7f4c11532640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4bf80a2f20 0x7f4bf80a53e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4bf402c8e0 tx=0x7f4bf4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.501+0000 7f4c11532640 1 -- 172.21.15.49:0/3536496131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c0c1023e0 msgr2=0x7f4c0c197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.501+0000 7f4c11532640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c0c1023e0 0x7f4c0c197ba0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00c920 tx=0x7f4bfc00cde0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.501+0000 7f4c11532640 1 -- 172.21.15.49:0/3536496131 shutdown_connections 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.501+0000 7f4c11532640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4bf80a2f20 0x7f4bf80a53e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.501+0000 7f4c11532640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c0c104d10 0x7f4c0c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.501+0000 7f4c11532640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c107640 0x7f4c0c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.501+0000 7f4c11532640 1 --2- 172.21.15.49:0/3536496131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c0c1023e0 0x7f4c0c197ba0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.502+0000 7f4c11532640 1 -- 172.21.15.49:0/3536496131 >> 172.21.15.49:0/3536496131 conn(0x7f4c0c0fbdf0 msgr2=0x7f4c0c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.502+0000 7f4c11532640 1 -- 172.21.15.49:0/3536496131 shutdown_connections 2024-03-26T17:59:36.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:36.502+0000 7f4c11532640 1 -- 172.21.15.49:0/3536496131 wait complete. 2024-03-26T17:59:36.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:36 smithi115 ceph-mon[36174]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:36.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:36 smithi115 ceph-mon[36174]: Deploying daemon mds.g on smithi049 2024-03-26T17:59:36.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:36 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] up:boot 2024-03-26T17:59:36.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:36 smithi115 ceph-mon[36174]: fsmap 9 up:standby 2024-03-26T17:59:36.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:36 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T17:59:36.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:36 smithi171 ceph-mon[36033]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:36.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:36 smithi171 ceph-mon[36033]: Deploying daemon mds.g on smithi049 2024-03-26T17:59:36.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:36 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] up:boot 2024-03-26T17:59:36.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:36 smithi171 ceph-mon[36033]: fsmap 9 up:standby 2024-03-26T17:59:36.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:36 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T17:59:37.112 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 17:59:36 smithi049 systemd[1]: Starting Ceph mds.g for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T17:59:37.389 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:37 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3536496131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:37.432 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:37 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3536496131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:37.433 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 17:59:37 smithi049 podman[73458]: 2024-03-26 17:59:37.088341809 +0000 UTC m=+0.018601600 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:59:37.433 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 17:59:37 smithi049 podman[73458]: 2024-03-26 17:59:37.222509997 +0000 UTC m=+0.152769786 container create 052ef4e36fe03b656a71f9e9c66636519da3d2022e05a655796d22d47c16e3b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T17:59:37.433 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 17:59:37 smithi049 podman[73458]: 2024-03-26 17:59:37.450239148 +0000 UTC m=+0.380498937 container init 052ef4e36fe03b656a71f9e9c66636519da3d2022e05a655796d22d47c16e3b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-26T17:59:37.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:37 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3536496131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:37.753 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 17:59:37 smithi049 podman[73458]: 2024-03-26 17:59:37.453405532 +0000 UTC m=+0.383665323 container start 052ef4e36fe03b656a71f9e9c66636519da3d2022e05a655796d22d47c16e3b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-26T17:59:37.753 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 17:59:37 smithi049 ceph-mds[73479]: -- 172.21.15.49:0/631984271 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c181b31a40 con 0x55c181b04c00 2024-03-26T17:59:37.753 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 17:59:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g[73474]: starting mds.g at 2024-03-26T17:59:37.753 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 17:59:37 smithi049 bash[73458]: 052ef4e36fe03b656a71f9e9c66636519da3d2022e05a655796d22d47c16e3b1 2024-03-26T17:59:37.753 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 17:59:37 smithi049 systemd[1]: Started Ceph mds.g for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:38 smithi049 ceph-mon[32331]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:38 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:38 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:38 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:38 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:38 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:38.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:38 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:38.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:38 smithi115 ceph-mon[36174]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:38.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:38 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:38.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:38 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:38.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:38 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:38.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:38 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:38.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:38 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:38.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:38 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:38.497 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:38.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:38 smithi171 ceph-mon[36033]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:38.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:38 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:38.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:38 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:38.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:38 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:38.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:38 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:38.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:38 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:38.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:38 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:38.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.629+0000 7f39c8f59640 1 -- 172.21.15.49:0/2716497228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c4103490 msgr2=0x7f39c4109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:38.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.629+0000 7f39c8f59640 1 --2- 172.21.15.49:0/2716497228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c4103490 0x7f39c4109d20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009ca0 tx=0x7f39bc02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:38.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.629+0000 7f39c8f59640 1 -- 172.21.15.49:0/2716497228 shutdown_connections 2024-03-26T17:59:38.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.630+0000 7f39c8f59640 1 --2- 172.21.15.49:0/2716497228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c4103490 0x7f39c4109d20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:38.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.630+0000 7f39c8f59640 1 --2- 172.21.15.49:0/2716497228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c4102ad0 0x7f39c4102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:38.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.630+0000 7f39c8f59640 1 --2- 172.21.15.49:0/2716497228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39c41018e0 0x7f39c4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:38.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.630+0000 7f39c8f59640 1 -- 172.21.15.49:0/2716497228 >> 172.21.15.49:0/2716497228 conn(0x7f39c40fce70 msgr2=0x7f39c40ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:38.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.630+0000 7f39c8f59640 1 -- 172.21.15.49:0/2716497228 shutdown_connections 2024-03-26T17:59:38.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.630+0000 7f39c8f59640 1 -- 172.21.15.49:0/2716497228 wait complete. 2024-03-26T17:59:38.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c8f59640 1 Processor -- start 2024-03-26T17:59:38.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c8f59640 1 -- start start 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c8f59640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39c41018e0 0x7f39c4075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c8f59640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c4102ad0 0x7f39c4073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c8f59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c4103490 0x7f39c40741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c8f59640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39c410d560 con 0x7f39c4103490 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c8f59640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39c410d3e0 con 0x7f39c4102ad0 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c8f59640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f39c410d6e0 con 0x7f39c41018e0 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c4103490 0x7f39c40741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c4103490 0x7f39c40741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41218/0 (socket says 172.21.15.49:41218) 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c2d9d640 1 -- 172.21.15.49:0/4113757008 learned_addr learned my addr 172.21.15.49:0/4113757008 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c259c640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39c41018e0 0x7f39c4075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c1d9b640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c4102ad0 0x7f39c4073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c2d9d640 1 -- 172.21.15.49:0/4113757008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39c41018e0 msgr2=0x7f39c4075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c2d9d640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39c41018e0 0x7f39c4075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:38.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c2d9d640 1 -- 172.21.15.49:0/4113757008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c4102ad0 msgr2=0x7f39c4073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c2d9d640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c4102ad0 0x7f39c4073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c2d9d640 1 -- 172.21.15.49:0/4113757008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c4074a80 con 0x7f39c4103490 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c259c640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39c41018e0 0x7f39c4075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.631+0000 7f39c1d9b640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c4102ad0 0x7f39c4073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.632+0000 7f39c2d9d640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c4103490 0x7f39c40741f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39bc02c700 tx=0x7f39bc032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.632+0000 7f39b37fe640 1 -- 172.21.15.49:0/4113757008 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc039040 con 0x7f39c4103490 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.632+0000 7f39c8f59640 1 -- 172.21.15.49:0/4113757008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c41a1c30 con 0x7f39c4103490 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.632+0000 7f39b37fe640 1 -- 172.21.15.49:0/4113757008 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39bc044070 con 0x7f39c4103490 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.632+0000 7f39b37fe640 1 -- 172.21.15.49:0/4113757008 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0042a0 con 0x7f39c4103490 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.632+0000 7f39c8f59640 1 -- 172.21.15.49:0/4113757008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c41a21d0 con 0x7f39c4103490 2024-03-26T17:59:38.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.633+0000 7f39c8f59640 1 -- 172.21.15.49:0/4113757008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c4108c00 con 0x7f39c4103490 2024-03-26T17:59:38.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.634+0000 7f39b37fe640 1 -- 172.21.15.49:0/4113757008 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f39bc003970 con 0x7f39c4103490 2024-03-26T17:59:38.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.635+0000 7f39b37fe640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39ac0a2f20 0x7f39ac0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:38.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.635+0000 7f39c259c640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39ac0a2f20 0x7f39ac0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:38.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.635+0000 7f39b37fe640 1 -- 172.21.15.49:0/4113757008 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f39bc0f1af0 con 0x7f39c4103490 2024-03-26T17:59:38.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.635+0000 7f39c259c640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39ac0a2f20 0x7f39ac0a53e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39b80058c0 tx=0x7f39b8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:38.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.639+0000 7f39b37fe640 1 -- 172.21.15.49:0/4113757008 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f39bc0bef70 con 0x7f39c4103490 2024-03-26T17:59:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.862+0000 7f39c8f59640 1 -- 172.21.15.49:0/4113757008 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39c402d240 con 0x7f39c4103490 2024-03-26T17:59:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.863+0000 7f39b37fe640 1 -- 172.21.15.49:0/4113757008 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f39bc0be870 con 0x7f39c4103490 2024-03-26T17:59:38.847 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:38.847 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-26T17:59:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 11 2024-03-26T17:59:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 -- 172.21.15.49:0/4113757008 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39ac0a2f20 msgr2=0x7f39ac0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39ac0a2f20 0x7f39ac0a53e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39b80058c0 tx=0x7f39b8005850 comp rx=0 tx=0).stop 2024-03-26T17:59:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 -- 172.21.15.49:0/4113757008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c4103490 msgr2=0x7f39c40741f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c4103490 0x7f39c40741f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39bc02c700 tx=0x7f39bc032f20 comp rx=0 tx=0).stop 2024-03-26T17:59:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 -- 172.21.15.49:0/4113757008 shutdown_connections 2024-03-26T17:59:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39ac0a2f20 0x7f39ac0a53e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39c4103490 0x7f39c40741f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39c4102ad0 0x7f39c4073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 --2- 172.21.15.49:0/4113757008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39c41018e0 0x7f39c4075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 -- 172.21.15.49:0/4113757008 >> 172.21.15.49:0/4113757008 conn(0x7f39c40fce70 msgr2=0x7f39c4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 -- 172.21.15.49:0/4113757008 shutdown_connections 2024-03-26T17:59:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:38.865+0000 7f39c8f59640 1 -- 172.21.15.49:0/4113757008 wait complete. 2024-03-26T17:59:39.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:39 smithi049 ceph-mon[32331]: Deploying daemon mds.a on smithi049 2024-03-26T17:59:39.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:39 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.49:6838/906046850,v1:172.21.15.49:6839/906046850] up:boot 2024-03-26T17:59:39.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:39 smithi049 ceph-mon[32331]: fsmap 10 up:standby 2024-03-26T17:59:39.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:39 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T17:59:39.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:39 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4113757008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:39.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:39 smithi115 ceph-mon[36174]: Deploying daemon mds.a on smithi049 2024-03-26T17:59:39.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:39 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.49:6838/906046850,v1:172.21.15.49:6839/906046850] up:boot 2024-03-26T17:59:39.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:39 smithi115 ceph-mon[36174]: fsmap 10 up:standby 2024-03-26T17:59:39.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:39 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T17:59:39.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:39 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4113757008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:39.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:39 smithi171 ceph-mon[36033]: Deploying daemon mds.a on smithi049 2024-03-26T17:59:39.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:39 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.49:6838/906046850,v1:172.21.15.49:6839/906046850] up:boot 2024-03-26T17:59:39.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:39 smithi171 ceph-mon[36033]: fsmap 10 up:standby 2024-03-26T17:59:39.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:39 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T17:59:39.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:39 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4113757008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:40.125 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 17:59:40 smithi049 podman[73822]: 2024-03-26 17:59:40.055862324 +0000 UTC m=+0.143646307 container create e76a9c150d1beacb54cb29078a5aa93cbf184c52ae32a2bdeccdf4f3316382e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-26T17:59:40.450 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:40 smithi049 ceph-mon[32331]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:40.450 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 17:59:40 smithi049 podman[73822]: 2024-03-26 17:59:40.350558587 +0000 UTC m=+0.438342567 container init e76a9c150d1beacb54cb29078a5aa93cbf184c52ae32a2bdeccdf4f3316382e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True) 2024-03-26T17:59:40.450 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 17:59:40 smithi049 podman[73822]: 2024-03-26 17:59:40.354038263 +0000 UTC m=+0.441822243 container start e76a9c150d1beacb54cb29078a5aa93cbf184c52ae32a2bdeccdf4f3316382e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T17:59:40.450 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 17:59:40 smithi049 ceph-mds[73841]: -- 172.21.15.49:0/326588813 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ee8c91a40 con 0x561ee8c64c00 2024-03-26T17:59:40.450 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 17:59:40 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a[73837]: starting mds.a at 2024-03-26T17:59:40.450 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 17:59:40 smithi049 bash[73822]: e76a9c150d1beacb54cb29078a5aa93cbf184c52ae32a2bdeccdf4f3316382e4 2024-03-26T17:59:40.488 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:40 smithi115 ceph-mon[36174]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:40.607 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:40 smithi171 ceph-mon[36033]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:40.753 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 17:59:40 smithi049 systemd[1]: Started Ceph mds.a for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:40.860 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:40.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.993+0000 7f96b784b640 1 -- 172.21.15.49:0/1215375042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96b01019f0 msgr2=0x7f96b0101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.993+0000 7f96b784b640 1 --2- 172.21.15.49:0/1215375042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96b01019f0 0x7f96b0101e70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009900 tx=0x7f96a002bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:40.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.993+0000 7f96b784b640 1 -- 172.21.15.49:0/1215375042 shutdown_connections 2024-03-26T17:59:40.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.993+0000 7f96b784b640 1 --2- 172.21.15.49:0/1215375042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b01023b0 0x7f96b0108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.993+0000 7f96b784b640 1 --2- 172.21.15.49:0/1215375042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96b01019f0 0x7f96b0101e70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.993+0000 7f96b784b640 1 --2- 172.21.15.49:0/1215375042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96b0100800 0x7f96b0100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.993+0000 7f96b784b640 1 -- 172.21.15.49:0/1215375042 >> 172.21.15.49:0/1215375042 conn(0x7f96b00fbd70 msgr2=0x7f96b00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:40.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.993+0000 7f96b784b640 1 -- 172.21.15.49:0/1215375042 shutdown_connections 2024-03-26T17:59:40.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.993+0000 7f96b784b640 1 -- 172.21.15.49:0/1215375042 wait complete. 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b784b640 1 Processor -- start 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b784b640 1 -- start start 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b784b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b0100800 0x7f96b006aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b784b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96b01019f0 0x7f96b019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b784b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96b01023b0 0x7f96b019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b784b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f96b010c480 con 0x7f96b01023b0 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b784b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f96b010c300 con 0x7f96b01019f0 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b784b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96b010c600 con 0x7f96b0100800 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b5de8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96b01023b0 0x7f96b019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b5de8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96b01023b0 0x7f96b019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41246/0 (socket says 172.21.15.49:41246) 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b4de6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96b01019f0 0x7f96b019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b5de8640 1 -- 172.21.15.49:0/950490696 learned_addr learned my addr 172.21.15.49:0/950490696 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b55e7640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b0100800 0x7f96b006aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b5de8640 1 -- 172.21.15.49:0/950490696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b0100800 msgr2=0x7f96b006aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b5de8640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b0100800 0x7f96b006aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b5de8640 1 -- 172.21.15.49:0/950490696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96b01019f0 msgr2=0x7f96b019c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b5de8640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96b01019f0 0x7f96b019c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b5de8640 1 -- 172.21.15.49:0/950490696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b019f860 con 0x7f96b01023b0 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b55e7640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b0100800 0x7f96b006aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b4de6640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96b01019f0 0x7f96b019c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.994+0000 7f96b5de8640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96b01023b0 0x7f96b019efa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f96a800cad0 tx=0x7f96a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.995+0000 7f96a67fc640 1 -- 172.21.15.49:0/950490696 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a800ae20 con 0x7f96b01023b0 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.995+0000 7f96b784b640 1 -- 172.21.15.49:0/950490696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b019fb50 con 0x7f96b01023b0 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.995+0000 7f96a67fc640 1 -- 172.21.15.49:0/950490696 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a8005020 con 0x7f96b01023b0 2024-03-26T17:59:40.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.995+0000 7f96a67fc640 1 -- 172.21.15.49:0/950490696 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a800f6b0 con 0x7f96b01023b0 2024-03-26T17:59:40.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.995+0000 7f96b784b640 1 -- 172.21.15.49:0/950490696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b01a0070 con 0x7f96b01023b0 2024-03-26T17:59:40.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.996+0000 7f96b784b640 1 -- 172.21.15.49:0/950490696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b0107b20 con 0x7f96b01023b0 2024-03-26T17:59:40.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.997+0000 7f96a67fc640 1 -- 172.21.15.49:0/950490696 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f96a80051d0 con 0x7f96b01023b0 2024-03-26T17:59:40.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.997+0000 7f96a67fc640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f969c0a2f20 0x7f969c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.997+0000 7f96b55e7640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f969c0a2f20 0x7f969c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.997+0000 7f96a67fc640 1 -- 172.21.15.49:0/950490696 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f96a80cf2e0 con 0x7f96b01023b0 2024-03-26T17:59:40.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:40.998+0000 7f96b55e7640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f969c0a2f20 0x7f969c0a53e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f96b0101850 tx=0x7f96ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:40.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.002+0000 7f96a67fc640 1 -- 172.21.15.49:0/950490696 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f96a8014030 con 0x7f96b01023b0 2024-03-26T17:59:41.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.228+0000 7f96b784b640 1 -- 172.21.15.49:0/950490696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96b002d3d0 con 0x7f96b01023b0 2024-03-26T17:59:41.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.229+0000 7f96a67fc640 1 -- 172.21.15.49:0/950490696 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f96a809c5b0 con 0x7f96b01023b0 2024-03-26T17:59:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:41.213 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-26T17:59:41.213 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 12 2024-03-26T17:59:41.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 -- 172.21.15.49:0/950490696 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f969c0a2f20 msgr2=0x7f969c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:41.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f969c0a2f20 0x7f969c0a53e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f96b0101850 tx=0x7f96ac005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 -- 172.21.15.49:0/950490696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96b01023b0 msgr2=0x7f96b019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96b01023b0 0x7f96b019efa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f96a800cad0 tx=0x7f96a800cf90 comp rx=0 tx=0).stop 2024-03-26T17:59:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 -- 172.21.15.49:0/950490696 shutdown_connections 2024-03-26T17:59:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f969c0a2f20 0x7f969c0a53e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f96b01023b0 0x7f96b019efa0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96b01019f0 0x7f96b019c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:41.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 --2- 172.21.15.49:0/950490696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b0100800 0x7f96b006aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:41.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 -- 172.21.15.49:0/950490696 >> 172.21.15.49:0/950490696 conn(0x7f96b00fbd70 msgr2=0x7f96b0104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:41.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 -- 172.21.15.49:0/950490696 shutdown_connections 2024-03-26T17:59:41.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:41.231+0000 7f96b784b640 1 -- 172.21.15.49:0/950490696 wait complete. 2024-03-26T17:59:41.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:41.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:41.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:41.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:41.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:41.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:41.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: Deploying daemon mds.b on smithi115 2024-03-26T17:59:41.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.49:6840/935071179,v1:172.21.15.49:6841/935071179] up:boot 2024-03-26T17:59:41.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: fsmap 11 up:standby 2024-03-26T17:59:41.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T17:59:41.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:41 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/950490696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:41.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:41.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:41.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:41.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:41.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:41.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:41.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: Deploying daemon mds.b on smithi115 2024-03-26T17:59:41.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.49:6840/935071179,v1:172.21.15.49:6841/935071179] up:boot 2024-03-26T17:59:41.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: fsmap 11 up:standby 2024-03-26T17:59:41.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T17:59:41.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:41 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/950490696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:41.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:41.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:41.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:41.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:59:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:59:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: Deploying daemon mds.b on smithi115 2024-03-26T17:59:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.49:6840/935071179,v1:172.21.15.49:6841/935071179] up:boot 2024-03-26T17:59:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: fsmap 11 up:standby 2024-03-26T17:59:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T17:59:41.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:41 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/950490696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:42.789 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:42 smithi115 ceph-mon[36174]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:42.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:42 smithi171 ceph-mon[36033]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:43.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:42 smithi049 ceph-mon[32331]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:43.226 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.365+0000 7f5a7caac640 1 -- 172.21.15.49:0/2392265573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a78100860 msgr2=0x7f5a78100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.365+0000 7f5a7caac640 1 --2- 172.21.15.49:0/2392265573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a78100860 0x7f5a78100c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009900 tx=0x7f5a6802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.365+0000 7f5a7caac640 1 -- 172.21.15.49:0/2392265573 shutdown_connections 2024-03-26T17:59:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.365+0000 7f5a7caac640 1 --2- 172.21.15.49:0/2392265573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a78101a50 0x7f5a78101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.365+0000 7f5a7caac640 1 --2- 172.21.15.49:0/2392265573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a78102410 0x7f5a78108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.365+0000 7f5a7caac640 1 --2- 172.21.15.49:0/2392265573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a78100860 0x7f5a78100c80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.365+0000 7f5a7caac640 1 -- 172.21.15.49:0/2392265573 >> 172.21.15.49:0/2392265573 conn(0x7f5a780fbdf0 msgr2=0x7f5a780fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.365+0000 7f5a7caac640 1 -- 172.21.15.49:0/2392265573 shutdown_connections 2024-03-26T17:59:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.365+0000 7f5a7caac640 1 -- 172.21.15.49:0/2392265573 wait complete. 2024-03-26T17:59:43.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.366+0000 7f5a7caac640 1 Processor -- start 2024-03-26T17:59:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.366+0000 7f5a7caac640 1 -- start start 2024-03-26T17:59:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.366+0000 7f5a7caac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a78100860 0x7f5a78197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.366+0000 7f5a7caac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a78101a50 0x7f5a781980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.366+0000 7f5a7caac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a78102410 0x7f5a7819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.366+0000 7f5a7caac640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810c440 con 0x7f5a78102410 2024-03-26T17:59:43.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.366+0000 7f5a7caac640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810c2c0 con 0x7f5a78100860 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.366+0000 7f5a7caac640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7810c5c0 con 0x7f5a78101a50 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a76d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a78102410 0x7f5a7819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a76d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a78102410 0x7f5a7819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41262/0 (socket says 172.21.15.49:41262) 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a76d9d640 1 -- 172.21.15.49:0/3187459034 learned_addr learned my addr 172.21.15.49:0/3187459034 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a7659c640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a78100860 0x7f5a78197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a6ffff640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a78101a50 0x7f5a781980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a76d9d640 1 -- 172.21.15.49:0/3187459034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a78101a50 msgr2=0x7f5a781980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a76d9d640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a78101a50 0x7f5a781980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a76d9d640 1 -- 172.21.15.49:0/3187459034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a78100860 msgr2=0x7f5a78197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a76d9d640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a78100860 0x7f5a78197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a76d9d640 1 -- 172.21.15.49:0/3187459034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7819f7d0 con 0x7f5a78102410 2024-03-26T17:59:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a7659c640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a78100860 0x7f5a78197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a76d9d640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a78102410 0x7f5a7819f160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000eb10 tx=0x7f5a7000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a6f7fe640 1 -- 172.21.15.49:0/3187459034 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000ce20 con 0x7f5a78102410 2024-03-26T17:59:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a6f7fe640 1 -- 172.21.15.49:0/3187459034 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a70005020 con 0x7f5a78102410 2024-03-26T17:59:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a7caac640 1 -- 172.21.15.49:0/3187459034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7819fac0 con 0x7f5a78102410 2024-03-26T17:59:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a6f7fe640 1 -- 172.21.15.49:0/3187459034 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70010690 con 0x7f5a78102410 2024-03-26T17:59:43.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.367+0000 7f5a7caac640 1 -- 172.21.15.49:0/3187459034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7819ffc0 con 0x7f5a78102410 2024-03-26T17:59:43.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.369+0000 7f5a7caac640 1 -- 172.21.15.49:0/3187459034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a781a02b0 con 0x7f5a78102410 2024-03-26T17:59:43.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.370+0000 7f5a6f7fe640 1 -- 172.21.15.49:0/3187459034 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f5a70005540 con 0x7f5a78102410 2024-03-26T17:59:43.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.370+0000 7f5a6f7fe640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5a640a2f20 0x7f5a640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:43.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.370+0000 7f5a7659c640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5a640a2f20 0x7f5a640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:43.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.370+0000 7f5a6f7fe640 1 -- 172.21.15.49:0/3187459034 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5a700cf4c0 con 0x7f5a78102410 2024-03-26T17:59:43.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.371+0000 7f5a7659c640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5a640a2f20 0x7f5a640a53e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5a68000c00 tx=0x7f5a68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:43.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.375+0000 7f5a6f7fe640 1 -- 172.21.15.49:0/3187459034 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5a7009c790 con 0x7f5a78102410 2024-03-26T17:59:43.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.612+0000 7f5a7caac640 1 -- 172.21.15.49:0/3187459034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a781a0480 con 0x7f5a78102410 2024-03-26T17:59:43.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.613+0000 7f5a6f7fe640 1 -- 172.21.15.49:0/3187459034 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7f5a7009c090 con 0x7f5a78102410 2024-03-26T17:59:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:43.596 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24533,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-26T17:59:43.597 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 13 2024-03-26T17:59:43.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 -- 172.21.15.49:0/3187459034 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5a640a2f20 msgr2=0x7f5a640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:43.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5a640a2f20 0x7f5a640a53e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5a68000c00 tx=0x7f5a68005fd0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 -- 172.21.15.49:0/3187459034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a78102410 msgr2=0x7f5a7819f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:43.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a78102410 0x7f5a7819f160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000eb10 tx=0x7f5a7000ef10 comp rx=0 tx=0).stop 2024-03-26T17:59:43.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 -- 172.21.15.49:0/3187459034 shutdown_connections 2024-03-26T17:59:43.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5a640a2f20 0x7f5a640a53e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a78101a50 0x7f5a781980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a78102410 0x7f5a7819f160 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 --2- 172.21.15.49:0/3187459034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a78100860 0x7f5a78197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 -- 172.21.15.49:0/3187459034 >> 172.21.15.49:0/3187459034 conn(0x7f5a780fbdf0 msgr2=0x7f5a780fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:43.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 -- 172.21.15.49:0/3187459034 shutdown_connections 2024-03-26T17:59:43.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.615+0000 7f5a7caac640 1 -- 172.21.15.49:0/3187459034 wait complete. 2024-03-26T17:59:43.609 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:59:43.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.740+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1857375442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd681007e0 msgr2=0x7efd68100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:43.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.740+0000 7efd6e5a9640 1 --2- 172.21.15.49:0/1857375442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd681007e0 0x7efd68100c00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efd58009900 tx=0x7efd5802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:43.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.740+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1857375442 shutdown_connections 2024-03-26T17:59:43.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.740+0000 7efd6e5a9640 1 --2- 172.21.15.49:0/1857375442 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd681019d0 0x7efd68101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.740+0000 7efd6e5a9640 1 --2- 172.21.15.49:0/1857375442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd68102390 0x7efd68108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.740+0000 7efd6e5a9640 1 --2- 172.21.15.49:0/1857375442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd681007e0 0x7efd68100c00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.740+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1857375442 >> 172.21.15.49:0/1857375442 conn(0x7efd680fbd50 msgr2=0x7efd680fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:43.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.741+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1857375442 shutdown_connections 2024-03-26T17:59:43.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.741+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1857375442 wait complete. 2024-03-26T17:59:43.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.741+0000 7efd6e5a9640 1 Processor -- start 2024-03-26T17:59:43.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.741+0000 7efd6e5a9640 1 -- start start 2024-03-26T17:59:43.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6e5a9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd681007e0 0x7efd68104770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:43.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6e5a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd681019d0 0x7efd68102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6e5a9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68102390 0x7efd681032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6e5a9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810c4a0 con 0x7efd68102390 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6e5a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810c320 con 0x7efd681019d0 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6e5a9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810c620 con 0x7efd681007e0 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6cb46640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68102390 0x7efd681032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6cb46640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68102390 0x7efd681032f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41280/0 (socket says 172.21.15.49:41280) 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6cb46640 1 -- 172.21.15.49:0/1671475885 learned_addr learned my addr 172.21.15.49:0/1671475885 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd67fff640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd681007e0 0x7efd68104770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd677fe640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd681019d0 0x7efd68102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6cb46640 1 -- 172.21.15.49:0/1671475885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd681007e0 msgr2=0x7efd68104770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6cb46640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd681007e0 0x7efd68104770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6cb46640 1 -- 172.21.15.49:0/1671475885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd681019d0 msgr2=0x7efd68102db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:43.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6cb46640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd681019d0 0x7efd68102db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6cb46640 1 -- 172.21.15.49:0/1671475885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd68103b80 con 0x7efd68102390 2024-03-26T17:59:43.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6cb46640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68102390 0x7efd681032f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efd6000eb10 tx=0x7efd6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:43.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd657fa640 1 -- 172.21.15.49:0/1671475885 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60004b20 con 0x7efd68102390 2024-03-26T17:59:43.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd657fa640 1 -- 172.21.15.49:0/1671475885 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd60004cc0 con 0x7efd68102390 2024-03-26T17:59:43.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1671475885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd681a1dc0 con 0x7efd68102390 2024-03-26T17:59:43.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.742+0000 7efd657fa640 1 -- 172.21.15.49:0/1671475885 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60010760 con 0x7efd68102390 2024-03-26T17:59:43.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.743+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1671475885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd681a2280 con 0x7efd68102390 2024-03-26T17:59:43.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.744+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1671475885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd681a2570 con 0x7efd68102390 2024-03-26T17:59:43.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.745+0000 7efd657fa640 1 -- 172.21.15.49:0/1671475885 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7efd60005020 con 0x7efd68102390 2024-03-26T17:59:43.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.746+0000 7efd657fa640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efd500a2f20 0x7efd500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:43.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.746+0000 7efd67fff640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efd500a2f20 0x7efd500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:43.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.746+0000 7efd657fa640 1 -- 172.21.15.49:0/1671475885 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efd600cf650 con 0x7efd68102390 2024-03-26T17:59:43.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.746+0000 7efd67fff640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efd500a2f20 0x7efd500a53e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd5802c240 tx=0x7efd5802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:43.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.750+0000 7efd657fa640 1 -- 172.21.15.49:0/1671475885 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efd6009c920 con 0x7efd68102390 2024-03-26T17:59:43.737 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 17:59:43 smithi115 systemd[1]: Started Ceph mds.b for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T17:59:43.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.936+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1671475885 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd681a27e0 con 0x7efd68102390 2024-03-26T17:59:43.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.937+0000 7efd657fa640 1 -- 172.21.15.49:0/1671475885 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18464 (secure 0 0 0) 0x7efd6009c220 con 0x7efd68102390 2024-03-26T17:59:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:43.920 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:00.651658+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:59:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1671475885 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efd500a2f20 msgr2=0x7efd500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efd500a2f20 0x7efd500a53e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd5802c240 tx=0x7efd5802df90 comp rx=0 tx=0).stop 2024-03-26T17:59:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1671475885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68102390 msgr2=0x7efd681032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68102390 0x7efd681032f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efd6000eb10 tx=0x7efd6000c2d0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1671475885 shutdown_connections 2024-03-26T17:59:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efd500a2f20 0x7efd500a53e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd68102390 0x7efd681032f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd681019d0 0x7efd68102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 --2- 172.21.15.49:0/1671475885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd681007e0 0x7efd68104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1671475885 >> 172.21.15.49:0/1671475885 conn(0x7efd680fbd50 msgr2=0x7efd680fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1671475885 shutdown_connections 2024-03-26T17:59:43.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:43.939+0000 7efd6e5a9640 1 -- 172.21.15.49:0/1671475885 wait complete. 2024-03-26T17:59:43.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:59:43.934 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:59:44.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.063+0000 7f4587bf4640 1 -- 172.21.15.49:0/1403066007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45801019b0 msgr2=0x7f4580101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.063+0000 7f4587bf4640 1 --2- 172.21.15.49:0/1403066007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45801019b0 0x7f4580101e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f457c009900 tx=0x7f457c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:44.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.063+0000 7f4587bf4640 1 -- 172.21.15.49:0/1403066007 shutdown_connections 2024-03-26T17:59:44.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.063+0000 7f4587bf4640 1 --2- 172.21.15.49:0/1403066007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4580102370 0x7f4580108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.063+0000 7f4587bf4640 1 --2- 172.21.15.49:0/1403066007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45801019b0 0x7f4580101e30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 --2- 172.21.15.49:0/1403066007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45801007c0 0x7f4580100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 -- 172.21.15.49:0/1403066007 >> 172.21.15.49:0/1403066007 conn(0x7f45800fbd50 msgr2=0x7f45800fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 -- 172.21.15.49:0/1403066007 shutdown_connections 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 -- 172.21.15.49:0/1403066007 wait complete. 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 Processor -- start 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 -- start start 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45801007c0 0x7f45800695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45801019b0 0x7f458019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4580102370 0x7f458019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f458010c420 con 0x7f45801019b0 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f458010c2a0 con 0x7f4580102370 2024-03-26T17:59:44.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f4587bf4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f458010c5a0 con 0x7f45801007c0 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f458518f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45801019b0 0x7f458019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f458518f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45801019b0 0x7f458019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41302/0 (socket says 172.21.15.49:41302) 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.064+0000 7f458518f640 1 -- 172.21.15.49:0/3225750518 learned_addr learned my addr 172.21.15.49:0/3225750518 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f4585990640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45801007c0 0x7f45800695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f4586191640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4580102370 0x7f458019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f458518f640 1 -- 172.21.15.49:0/3225750518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45801007c0 msgr2=0x7f45800695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f458518f640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45801007c0 0x7f45800695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f458518f640 1 -- 172.21.15.49:0/3225750518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4580102370 msgr2=0x7f458019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f458518f640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4580102370 0x7f458019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f458518f640 1 -- 172.21.15.49:0/3225750518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458019f810 con 0x7f45801019b0 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f458518f640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45801019b0 0x7f458019c8e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f457c009230 tx=0x7f457c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f4576ffd640 1 -- 172.21.15.49:0/3225750518 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c041070 con 0x7f45801019b0 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f4576ffd640 1 -- 172.21.15.49:0/3225750518 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c0044d0 con 0x7f45801019b0 2024-03-26T17:59:44.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f4587bf4640 1 -- 172.21.15.49:0/3225750518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458019faa0 con 0x7f45801019b0 2024-03-26T17:59:44.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f4576ffd640 1 -- 172.21.15.49:0/3225750518 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c034440 con 0x7f45801019b0 2024-03-26T17:59:44.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.065+0000 7f4587bf4640 1 -- 172.21.15.49:0/3225750518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458019ff90 con 0x7f45801019b0 2024-03-26T17:59:44.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.066+0000 7f4587bf4640 1 -- 172.21.15.49:0/3225750518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4580107b30 con 0x7f45801019b0 2024-03-26T17:59:44.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.068+0000 7f4576ffd640 1 -- 172.21.15.49:0/3225750518 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f457c044070 con 0x7f45801019b0 2024-03-26T17:59:44.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.069+0000 7f4576ffd640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f456c0a2f20 0x7f456c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.069+0000 7f4585990640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f456c0a2f20 0x7f456c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.069+0000 7f4576ffd640 1 -- 172.21.15.49:0/3225750518 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f457c0f0b20 con 0x7f45801019b0 2024-03-26T17:59:44.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.069+0000 7f4585990640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f456c0a2f20 0x7f456c0a53e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4580101810 tx=0x7f4570005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:44.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.074+0000 7f4576ffd640 1 -- 172.21.15.49:0/3225750518 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f457c0bd7a0 con 0x7f45801019b0 2024-03-26T17:59:44.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.270+0000 7f4587bf4640 1 -- 172.21.15.49:0/3225750518 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f458002d490 con 0x7f45801019b0 2024-03-26T17:59:44.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.270+0000 7f4576ffd640 1 -- 172.21.15.49:0/3225750518 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f457c0c1840 con 0x7f45801019b0 2024-03-26T17:59:44.253 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T17:59:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.272+0000 7f4587bf4640 1 -- 172.21.15.49:0/3225750518 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f456c0a2f20 msgr2=0x7f456c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.272+0000 7f4587bf4640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f456c0a2f20 0x7f456c0a53e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4580101810 tx=0x7f4570005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.272+0000 7f4587bf4640 1 -- 172.21.15.49:0/3225750518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45801019b0 msgr2=0x7f458019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.272+0000 7f4587bf4640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45801019b0 0x7f458019c8e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f457c009230 tx=0x7f457c005d50 comp rx=0 tx=0).stop 2024-03-26T17:59:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.272+0000 7f4587bf4640 1 -- 172.21.15.49:0/3225750518 shutdown_connections 2024-03-26T17:59:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.272+0000 7f4587bf4640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f456c0a2f20 0x7f456c0a53e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.272+0000 7f4587bf4640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4580102370 0x7f458019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.272+0000 7f4587bf4640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f45801019b0 0x7f458019c8e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.272+0000 7f4587bf4640 1 --2- 172.21.15.49:0/3225750518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45801007c0 0x7f45800695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.273+0000 7f4587bf4640 1 -- 172.21.15.49:0/3225750518 >> 172.21.15.49:0/3225750518 conn(0x7f45800fbd50 msgr2=0x7f4580104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.273+0000 7f4587bf4640 1 -- 172.21.15.49:0/3225750518 shutdown_connections 2024-03-26T17:59:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.273+0000 7f4587bf4640 1 -- 172.21.15.49:0/3225750518 wait complete. 2024-03-26T17:59:44.268 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-26T17:59:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 -- 172.21.15.49:0/610945937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabec102390 msgr2=0x7fabec108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 --2- 172.21.15.49:0/610945937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabec102390 0x7fabec108c20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009ca0 tx=0x7fabe402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 -- 172.21.15.49:0/610945937 shutdown_connections 2024-03-26T17:59:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 --2- 172.21.15.49:0/610945937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabec102390 0x7fabec108c20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 --2- 172.21.15.49:0/610945937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabec1019d0 0x7fabec101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 --2- 172.21.15.49:0/610945937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabec1007e0 0x7fabec100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 -- 172.21.15.49:0/610945937 >> 172.21.15.49:0/610945937 conn(0x7fabec0fbd50 msgr2=0x7fabec0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 -- 172.21.15.49:0/610945937 shutdown_connections 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 -- 172.21.15.49:0/610945937 wait complete. 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 Processor -- start 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 -- start start 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabec1007e0 0x7fabec1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabec1019d0 0x7fabec196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabec102390 0x7fabec19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fabec10c440 con 0x7fabec1019d0 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fabec10c2c0 con 0x7fabec1007e0 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.396+0000 7fabf2df3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fabec10c5c0 con 0x7fabec102390 2024-03-26T17:59:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabec1019d0 0x7fabec196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabec1019d0 0x7fabec196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41332/0 (socket says 172.21.15.49:41332) 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe3fff640 1 -- 172.21.15.49:0/2275088904 learned_addr learned my addr 172.21.15.49:0/2275088904 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabf1390640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabec102390 0x7fabec19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabf0b8f640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabec1007e0 0x7fabec1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe3fff640 1 -- 172.21.15.49:0/2275088904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabec102390 msgr2=0x7fabec19df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe3fff640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabec102390 0x7fabec19df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe3fff640 1 -- 172.21.15.49:0/2275088904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabec1007e0 msgr2=0x7fabec1969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe3fff640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabec1007e0 0x7fabec1969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe3fff640 1 -- 172.21.15.49:0/2275088904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabec19e5f0 con 0x7fabec1019d0 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabf1390640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabec102390 0x7fabec19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe3fff640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabec1019d0 0x7fabec196f10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00d950 tx=0x7fabdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe1ffb640 1 -- 172.21.15.49:0/2275088904 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc017030 con 0x7fabec1019d0 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe1ffb640 1 -- 172.21.15.49:0/2275088904 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabdc004020 con 0x7fabec1019d0 2024-03-26T17:59:44.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabe1ffb640 1 -- 172.21.15.49:0/2275088904 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0047a0 con 0x7fabec1019d0 2024-03-26T17:59:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabf2df3640 1 -- 172.21.15.49:0/2275088904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabec19e8e0 con 0x7fabec1019d0 2024-03-26T17:59:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.397+0000 7fabf2df3640 1 -- 172.21.15.49:0/2275088904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabec19ee80 con 0x7fabec1019d0 2024-03-26T17:59:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.398+0000 7fabf2df3640 1 -- 172.21.15.49:0/2275088904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabec107a30 con 0x7fabec1019d0 2024-03-26T17:59:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.399+0000 7fabe1ffb640 1 -- 172.21.15.49:0/2275088904 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fabdc0041d0 con 0x7fabec1019d0 2024-03-26T17:59:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.400+0000 7fabe1ffb640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fabd40a2fa0 0x7fabd40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.400+0000 7fabf0b8f640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fabd40a2fa0 0x7fabd40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.400+0000 7fabe1ffb640 1 -- 172.21.15.49:0/2275088904 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fabdc0ce8d0 con 0x7fabec1019d0 2024-03-26T17:59:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.400+0000 7fabf0b8f640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fabd40a2fa0 0x7fabd40a5460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabec101830 tx=0x7fabe8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:44.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.404+0000 7fabe1ffb640 1 -- 172.21.15.49:0/2275088904 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fabdc09bba0 con 0x7fabec1019d0 2024-03-26T17:59:44.567 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:44.567 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.567 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.567 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.567 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.567 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] up:boot 2024-03-26T17:59:44.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: fsmap 12 up:standby 2024-03-26T17:59:44.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T17:59:44.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3187459034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:44.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:44.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1671475885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:44.568 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:44 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3225750518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:59:44.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.596+0000 7fabf2df3640 1 -- 172.21.15.49:0/2275088904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fabec02d240 con 0x7fabec1019d0 2024-03-26T17:59:44.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.596+0000 7fabe1ffb640 1 -- 172.21.15.49:0/2275088904 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fabdc09b4a0 con 0x7fabec1019d0 2024-03-26T17:59:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:44.579 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2024-03-26T17:59:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T17:59:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2024-03-26T17:59:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2024-03-26T17:59:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T17:59:44.580 INFO:teuthology.orchestra.run.smithi049.stdout:] 2024-03-26T17:59:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 -- 172.21.15.49:0/2275088904 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fabd40a2fa0 msgr2=0x7fabd40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fabd40a2fa0 0x7fabd40a5460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabec101830 tx=0x7fabe8005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 -- 172.21.15.49:0/2275088904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabec1019d0 msgr2=0x7fabec196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabec1019d0 0x7fabec196f10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00d950 tx=0x7fabdc00de10 comp rx=0 tx=0).stop 2024-03-26T17:59:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 -- 172.21.15.49:0/2275088904 shutdown_connections 2024-03-26T17:59:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fabd40a2fa0 0x7fabd40a5460 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabec102390 0x7fabec19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fabec1019d0 0x7fabec196f10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 --2- 172.21.15.49:0/2275088904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabec1007e0 0x7fabec1969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 -- 172.21.15.49:0/2275088904 >> 172.21.15.49:0/2275088904 conn(0x7fabec0fbd50 msgr2=0x7fabec104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 -- 172.21.15.49:0/2275088904 shutdown_connections 2024-03-26T17:59:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.599+0000 7fabf2df3640 1 -- 172.21.15.49:0/2275088904 wait complete. 2024-03-26T17:59:44.596 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-26T17:59:44.596 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-26T17:59:44.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:44.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] up:boot 2024-03-26T17:59:44.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: fsmap 12 up:standby 2024-03-26T17:59:44.695 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T17:59:44.695 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3187459034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:44.695 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:44.695 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1671475885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:44.695 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:44 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3225750518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:59:44.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.726+0000 7fdaa0436640 1 -- 172.21.15.49:0/271090527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda98107600 msgr2=0x7fda981099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.726+0000 7fdaa0436640 1 --2- 172.21.15.49:0/271090527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda98107600 0x7fda981099f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda90009ca0 tx=0x7fda9002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.726+0000 7fdaa0436640 1 -- 172.21.15.49:0/271090527 shutdown_connections 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.726+0000 7fdaa0436640 1 --2- 172.21.15.49:0/271090527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda98107600 0x7fda981099f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.726+0000 7fdaa0436640 1 --2- 172.21.15.49:0/271090527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98104cd0 0x7fda981070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.726+0000 7fdaa0436640 1 --2- 172.21.15.49:0/271090527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda981023a0 0x7fda98104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 -- 172.21.15.49:0/271090527 >> 172.21.15.49:0/271090527 conn(0x7fda980fbd50 msgr2=0x7fda980fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 -- 172.21.15.49:0/271090527 shutdown_connections 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 -- 172.21.15.49:0/271090527 wait complete. 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 Processor -- start 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 -- start start 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda981023a0 0x7fda98197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98104cd0 0x7fda98198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda98107600 0x7fda9819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fda9810d540 con 0x7fda981023a0 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fda9810d3c0 con 0x7fda98104cd0 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fdaa0436640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fda9810d6c0 con 0x7fda98107600 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fda9e1d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda981023a0 0x7fda98197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fda9d9d1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98104cd0 0x7fda98198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fda9e1d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda981023a0 0x7fda98197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41352/0 (socket says 172.21.15.49:41352) 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.727+0000 7fda9e1d2640 1 -- 172.21.15.49:0/1062065710 learned_addr learned my addr 172.21.15.49:0/1062065710 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda9e9d3640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda98107600 0x7fda9819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda9d9d1640 1 -- 172.21.15.49:0/1062065710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda98107600 msgr2=0x7fda9819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda9d9d1640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda98107600 0x7fda9819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda9d9d1640 1 -- 172.21.15.49:0/1062065710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda981023a0 msgr2=0x7fda98197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:44.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda9d9d1640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda981023a0 0x7fda98197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda9d9d1640 1 -- 172.21.15.49:0/1062065710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9819f770 con 0x7fda98104cd0 2024-03-26T17:59:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda9e1d2640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda981023a0 0x7fda98197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda9d9d1640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98104cd0 0x7fda98198090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda88005fd0 tx=0x7fda8800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda8f7fe640 1 -- 172.21.15.49:0/1062065710 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88009c00 con 0x7fda98104cd0 2024-03-26T17:59:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fdaa0436640 1 -- 172.21.15.49:0/1062065710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9819fa60 con 0x7fda98104cd0 2024-03-26T17:59:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda8f7fe640 1 -- 172.21.15.49:0/1062065710 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda88009da0 con 0x7fda98104cd0 2024-03-26T17:59:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fda8f7fe640 1 -- 172.21.15.49:0/1062065710 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88010460 con 0x7fda98104cd0 2024-03-26T17:59:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.728+0000 7fdaa0436640 1 -- 172.21.15.49:0/1062065710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda981a0000 con 0x7fda98104cd0 2024-03-26T17:59:44.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.729+0000 7fdaa0436640 1 -- 172.21.15.49:0/1062065710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda980fd840 con 0x7fda98104cd0 2024-03-26T17:59:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.730+0000 7fda8f7fe640 1 -- 172.21.15.49:0/1062065710 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fda88003f50 con 0x7fda98104cd0 2024-03-26T17:59:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.731+0000 7fda8f7fe640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fda840a2f20 0x7fda840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.731+0000 7fda9e1d2640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fda840a2f20 0x7fda840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.731+0000 7fda8f7fe640 1 -- 172.21.15.49:0/1062065710 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fda880ceaa0 con 0x7fda98104cd0 2024-03-26T17:59:44.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.731+0000 7fda9e1d2640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fda840a2f20 0x7fda840a53e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda9400f4d0 tx=0x7fda9400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:44.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.734+0000 7fda8f7fe640 1 -- 172.21.15.49:0/1062065710 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fda88017020 con 0x7fda98104cd0 2024-03-26T17:59:44.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] up:boot 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: fsmap 12 up:standby 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3187459034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:44.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1671475885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:44.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:44 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3225750518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:59:44.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:44.919+0000 7fdaa0436640 1 -- 172.21.15.49:0/1062065710 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fda9802d240 con 0x7fda98104cd0 2024-03-26T17:59:45.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.416+0000 7fda8f7fe640 1 -- 172.21.15.49:0/1062065710 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fda8809bd70 con 0x7fda98104cd0 2024-03-26T17:59:45.399 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 -- 172.21.15.49:0/1062065710 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fda840a2f20 msgr2=0x7fda840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fda840a2f20 0x7fda840a53e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda9400f4d0 tx=0x7fda9400a380 comp rx=0 tx=0).stop 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 -- 172.21.15.49:0/1062065710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98104cd0 msgr2=0x7fda98198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98104cd0 0x7fda98198090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda88005fd0 tx=0x7fda8800dde0 comp rx=0 tx=0).stop 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 -- 172.21.15.49:0/1062065710 shutdown_connections 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fda840a2f20 0x7fda840a53e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda98107600 0x7fda9819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98104cd0 0x7fda98198090 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 --2- 172.21.15.49:0/1062065710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda981023a0 0x7fda98197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 -- 172.21.15.49:0/1062065710 >> 172.21.15.49:0/1062065710 conn(0x7fda980fbd50 msgr2=0x7fda98102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:45.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 -- 172.21.15.49:0/1062065710 shutdown_connections 2024-03-26T17:59:45.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.420+0000 7fdaa0436640 1 -- 172.21.15.49:0/1062065710 wait complete. 2024-03-26T17:59:45.415 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-26T17:59:45.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.547+0000 7fb598bc6640 1 -- 172.21.15.49:0/1677555069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb594102360 msgr2=0x7fb594104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:45.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.547+0000 7fb598bc6640 1 --2- 172.21.15.49:0/1677555069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb594102360 0x7fb594104750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb584009900 tx=0x7fb58402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.547+0000 7fb598bc6640 1 -- 172.21.15.49:0/1677555069 shutdown_connections 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.547+0000 7fb598bc6640 1 --2- 172.21.15.49:0/1677555069 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5941075c0 0x7fb594109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.547+0000 7fb598bc6640 1 --2- 172.21.15.49:0/1677555069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb594104c90 0x7fb594107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.547+0000 7fb598bc6640 1 --2- 172.21.15.49:0/1677555069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb594102360 0x7fb594104750 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.547+0000 7fb598bc6640 1 -- 172.21.15.49:0/1677555069 >> 172.21.15.49:0/1677555069 conn(0x7fb5940fbd70 msgr2=0x7fb5940fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.547+0000 7fb598bc6640 1 -- 172.21.15.49:0/1677555069 shutdown_connections 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.547+0000 7fb598bc6640 1 -- 172.21.15.49:0/1677555069 wait complete. 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.548+0000 7fb598bc6640 1 Processor -- start 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.548+0000 7fb598bc6640 1 -- start start 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.548+0000 7fb598bc6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb594102360 0x7fb594101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.548+0000 7fb598bc6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb594104c90 0x7fb594100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.548+0000 7fb598bc6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5941075c0 0x7fb594100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:45.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.548+0000 7fb598bc6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb59410d580 con 0x7fb594102360 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.548+0000 7fb598bc6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb59410d400 con 0x7fb594104c90 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.548+0000 7fb598bc6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb59410d700 con 0x7fb5941075c0 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.548+0000 7fb59259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb594102360 0x7fb594101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.548+0000 7fb59259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb594102360 0x7fb594101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41360/0 (socket says 172.21.15.49:41360) 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb59259c640 1 -- 172.21.15.49:0/1885542751 learned_addr learned my addr 172.21.15.49:0/1885542751 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb592d9d640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5941075c0 0x7fb594100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb591d9b640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb594104c90 0x7fb594100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb59259c640 1 -- 172.21.15.49:0/1885542751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5941075c0 msgr2=0x7fb594100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb59259c640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5941075c0 0x7fb594100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb59259c640 1 -- 172.21.15.49:0/1885542751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb594104c90 msgr2=0x7fb594100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb59259c640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb594104c90 0x7fb594100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:45.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb59259c640 1 -- 172.21.15.49:0/1885542751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5941010e0 con 0x7fb594102360 2024-03-26T17:59:45.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb59259c640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb594102360 0x7fb594101cd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb584009230 tx=0x7fb58402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:45.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb5837fe640 1 -- 172.21.15.49:0/1885542751 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584041070 con 0x7fb594102360 2024-03-26T17:59:45.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb598bc6640 1 -- 172.21.15.49:0/1885542751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5941a1ce0 con 0x7fb594102360 2024-03-26T17:59:45.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb5837fe640 1 -- 172.21.15.49:0/1885542751 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb584004450 con 0x7fb594102360 2024-03-26T17:59:45.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb5837fe640 1 -- 172.21.15.49:0/1885542751 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584003e20 con 0x7fb594102360 2024-03-26T17:59:45.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.549+0000 7fb598bc6640 1 -- 172.21.15.49:0/1885542751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5941a21d0 con 0x7fb594102360 2024-03-26T17:59:45.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.550+0000 7fb598bc6640 1 -- 172.21.15.49:0/1885542751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5940fd860 con 0x7fb594102360 2024-03-26T17:59:45.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.551+0000 7fb5837fe640 1 -- 172.21.15.49:0/1885542751 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fb584044070 con 0x7fb594102360 2024-03-26T17:59:45.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.552+0000 7fb5837fe640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb57c0a2f20 0x7fb57c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:45.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.552+0000 7fb591d9b640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb57c0a2f20 0x7fb57c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:45.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.552+0000 7fb5837fe640 1 -- 172.21.15.49:0/1885542751 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7fb5840f0c20 con 0x7fb594102360 2024-03-26T17:59:45.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.553+0000 7fb591d9b640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb57c0a2f20 0x7fb57c0a53e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb594101a40 tx=0x7fb588005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:45.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.557+0000 7fb5837fe640 1 -- 172.21.15.49:0/1885542751 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb5840bd730 con 0x7fb594102360 2024-03-26T17:59:45.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:45 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2275088904' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T17:59:45.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:45 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1062065710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:59:45.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:45 smithi115 ceph-mon[36174]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:59:45.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:45 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2275088904' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T17:59:45.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:45 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1062065710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:59:45.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:45 smithi049 ceph-mon[32331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:59:45.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:45.775+0000 7fb598bc6640 1 -- 172.21.15.49:0/1885542751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fb5940624c0 con 0x7fb594102360 2024-03-26T17:59:45.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:45 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2275088904' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T17:59:45.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:45 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1062065710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:59:45.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:45 smithi171 ceph-mon[36033]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:59:45.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:45.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:46.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.417+0000 7fb5837fe640 1 -- 172.21.15.49:0/1885542751 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7fb5840c17d0 con 0x7fb594102360 2024-03-26T17:59:46.400 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2024-03-26T17:59:46.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.419+0000 7fb598bc6640 1 -- 172.21.15.49:0/1885542751 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb57c0a2f20 msgr2=0x7fb57c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:46.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.419+0000 7fb598bc6640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb57c0a2f20 0x7fb57c0a53e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb594101a40 tx=0x7fb588005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:46.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.419+0000 7fb598bc6640 1 -- 172.21.15.49:0/1885542751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb594102360 msgr2=0x7fb594101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:46.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.419+0000 7fb598bc6640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb594102360 0x7fb594101cd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb584009230 tx=0x7fb58402c650 comp rx=0 tx=0).stop 2024-03-26T17:59:46.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.420+0000 7fb598bc6640 1 -- 172.21.15.49:0/1885542751 shutdown_connections 2024-03-26T17:59:46.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.420+0000 7fb598bc6640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb57c0a2f20 0x7fb57c0a53e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:46.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.420+0000 7fb598bc6640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5941075c0 0x7fb594100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:46.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.420+0000 7fb598bc6640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb594104c90 0x7fb594100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:46.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.420+0000 7fb598bc6640 1 --2- 172.21.15.49:0/1885542751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb594102360 0x7fb594101cd0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb584009230 tx=0x7fb58402c650 comp rx=0 tx=0).stop 2024-03-26T17:59:46.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.420+0000 7fb598bc6640 1 -- 172.21.15.49:0/1885542751 >> 172.21.15.49:0/1885542751 conn(0x7fb5940fbd70 msgr2=0x7fb594102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:46.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.420+0000 7fb598bc6640 1 -- 172.21.15.49:0/1885542751 shutdown_connections 2024-03-26T17:59:46.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.420+0000 7fb598bc6640 1 -- 172.21.15.49:0/1885542751 wait complete. 2024-03-26T17:59:46.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-26T17:59:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.556+0000 7f4e79201640 1 -- 172.21.15.49:0/600088860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e741006d0 msgr2=0x7f4e74100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.556+0000 7f4e79201640 1 --2- 172.21.15.49:0/600088860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e741006d0 0x7f4e74100af0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009900 tx=0x7f4e6402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.556+0000 7f4e79201640 1 -- 172.21.15.49:0/600088860 shutdown_connections 2024-03-26T17:59:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.556+0000 7f4e79201640 1 --2- 172.21.15.49:0/600088860 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e741018c0 0x7f4e74101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.556+0000 7f4e79201640 1 --2- 172.21.15.49:0/600088860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e74102280 0x7f4e74108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.556+0000 7f4e79201640 1 --2- 172.21.15.49:0/600088860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e741006d0 0x7f4e74100af0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.556+0000 7f4e79201640 1 -- 172.21.15.49:0/600088860 >> 172.21.15.49:0/600088860 conn(0x7f4e740fbc60 msgr2=0x7f4e740fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.556+0000 7f4e79201640 1 -- 172.21.15.49:0/600088860 shutdown_connections 2024-03-26T17:59:46.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.556+0000 7f4e79201640 1 -- 172.21.15.49:0/600088860 wait complete. 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e79201640 1 Processor -- start 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e79201640 1 -- start start 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e79201640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e741006d0 0x7f4e740695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e79201640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e741018c0 0x7f4e7419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e79201640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e74102280 0x7f4e7419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e79201640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7410c320 con 0x7f4e741006d0 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e79201640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7410c1a0 con 0x7f4e74102280 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e79201640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7410c4a0 con 0x7f4e741018c0 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e72d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e741006d0 0x7f4e740695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e72d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e741006d0 0x7f4e740695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41390/0 (socket says 172.21.15.49:41390) 2024-03-26T17:59:46.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e72d9d640 1 -- 172.21.15.49:0/3399640560 learned_addr learned my addr 172.21.15.49:0/3399640560 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e7359e640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e74102280 0x7f4e7419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.557+0000 7f4e7259c640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e741018c0 0x7f4e7419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e72d9d640 1 -- 172.21.15.49:0/3399640560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e741018c0 msgr2=0x7f4e7419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e72d9d640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e741018c0 0x7f4e7419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e72d9d640 1 -- 172.21.15.49:0/3399640560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e74102280 msgr2=0x7f4e7419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e72d9d640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e74102280 0x7f4e7419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e72d9d640 1 -- 172.21.15.49:0/3399640560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7419f710 con 0x7f4e741006d0 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e7359e640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e74102280 0x7f4e7419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e72d9d640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e741006d0 0x7f4e740695f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e64005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e5bfff640 1 -- 172.21.15.49:0/3399640560 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64041070 con 0x7f4e741006d0 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e5bfff640 1 -- 172.21.15.49:0/3399640560 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e640044d0 con 0x7f4e741006d0 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e79201640 1 -- 172.21.15.49:0/3399640560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7419f9a0 con 0x7f4e741006d0 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e5bfff640 1 -- 172.21.15.49:0/3399640560 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64034440 con 0x7f4e741006d0 2024-03-26T17:59:46.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.558+0000 7f4e79201640 1 -- 172.21.15.49:0/3399640560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7419fe90 con 0x7f4e741006d0 2024-03-26T17:59:46.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.559+0000 7f4e79201640 1 -- 172.21.15.49:0/3399640560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e74107a90 con 0x7f4e741006d0 2024-03-26T17:59:46.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.560+0000 7f4e5bfff640 1 -- 172.21.15.49:0/3399640560 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f4e64044070 con 0x7f4e741006d0 2024-03-26T17:59:46.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.561+0000 7f4e5bfff640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4e5c0a2f20 0x7f4e5c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:46.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.561+0000 7f4e7259c640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4e5c0a2f20 0x7f4e5c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:46.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.562+0000 7f4e5bfff640 1 -- 172.21.15.49:0/3399640560 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f4e640f1110 con 0x7f4e741006d0 2024-03-26T17:59:46.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.562+0000 7f4e7259c640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4e5c0a2f20 0x7f4e5c0a53e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005fd0 tx=0x7f4e68005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:46.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.565+0000 7f4e5bfff640 1 -- 172.21.15.49:0/3399640560 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4e640be050 con 0x7f4e741006d0 2024-03-26T17:59:46.699 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:46 smithi171 ceph-mon[36033]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:46.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:46.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:46.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:46 smithi171 ceph-mon[36033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T17:59:46.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:46 smithi171 ceph-mon[36033]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T17:59:46.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:46.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:46.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:46 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:46.700 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:46 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1885542751' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-26T17:59:46.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:46 smithi115 ceph-mon[36174]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:46.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:46.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:46.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:46 smithi115 ceph-mon[36174]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T17:59:46.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:46 smithi115 ceph-mon[36174]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T17:59:46.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:46.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:46.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:46 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:46.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:46 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1885542751' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-26T17:59:46.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:46 smithi049 ceph-mon[32331]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:46.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:46.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:46.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:46 smithi049 ceph-mon[32331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T17:59:46.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:46 smithi049 ceph-mon[32331]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T17:59:46.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:46.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:46.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:46 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:46.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:46 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1885542751' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-26T17:59:46.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:46.795+0000 7f4e79201640 1 -- 172.21.15.49:0/3399640560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4e74062530 con 0x7f4e741006d0 2024-03-26T17:59:47.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.423+0000 7f4e5bfff640 1 -- 172.21.15.49:0/3399640560 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f4e640bd950 con 0x7f4e741006d0 2024-03-26T17:59:47.406 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 2 and data pool 3 2024-03-26T17:59:47.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 -- 172.21.15.49:0/3399640560 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4e5c0a2f20 msgr2=0x7f4e5c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:47.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4e5c0a2f20 0x7f4e5c0a53e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005fd0 tx=0x7f4e68005850 comp rx=0 tx=0).stop 2024-03-26T17:59:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 -- 172.21.15.49:0/3399640560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e741006d0 msgr2=0x7f4e740695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e741006d0 0x7f4e740695f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4e64000c00 tx=0x7f4e64005d50 comp rx=0 tx=0).stop 2024-03-26T17:59:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 -- 172.21.15.49:0/3399640560 shutdown_connections 2024-03-26T17:59:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4e5c0a2f20 0x7f4e5c0a53e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e741018c0 0x7f4e7419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e74102280 0x7f4e7419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 --2- 172.21.15.49:0/3399640560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e741006d0 0x7f4e740695f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:47.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 -- 172.21.15.49:0/3399640560 >> 172.21.15.49:0/3399640560 conn(0x7f4e740fbc60 msgr2=0x7f4e74104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 -- 172.21.15.49:0/3399640560 shutdown_connections 2024-03-26T17:59:47.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.425+0000 7f4e79201640 1 -- 172.21.15.49:0/3399640560 wait complete. 2024-03-26T17:59:47.421 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-26T17:59:47.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-26T17:59:47.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.553+0000 7ff03e812640 1 -- 172.21.15.49:0/3015793211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038107640 msgr2=0x7ff038109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.553+0000 7ff03e812640 1 --2- 172.21.15.49:0/3015793211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038107640 0x7ff038109a30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff030009ca0 tx=0x7ff03002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.553+0000 7ff03e812640 1 -- 172.21.15.49:0/3015793211 shutdown_connections 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.553+0000 7ff03e812640 1 --2- 172.21.15.49:0/3015793211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038107640 0x7ff038109a30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.553+0000 7ff03e812640 1 --2- 172.21.15.49:0/3015793211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff038104d10 0x7ff038107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.553+0000 7ff03e812640 1 --2- 172.21.15.49:0/3015793211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0381023e0 0x7ff0381047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.553+0000 7ff03e812640 1 -- 172.21.15.49:0/3015793211 >> 172.21.15.49:0/3015793211 conn(0x7ff0380fbd50 msgr2=0x7ff0380fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.554+0000 7ff03e812640 1 -- 172.21.15.49:0/3015793211 shutdown_connections 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.554+0000 7ff03e812640 1 -- 172.21.15.49:0/3015793211 wait complete. 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.554+0000 7ff03e812640 1 Processor -- start 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.554+0000 7ff03e812640 1 -- start start 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.554+0000 7ff03e812640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0381023e0 0x7ff038197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:47.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.554+0000 7ff03e812640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff038104d10 0x7ff038198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.554+0000 7ff03e812640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038107640 0x7ff03819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.554+0000 7ff03e812640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff03810d560 con 0x7ff038107640 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.554+0000 7ff03e812640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff03810d3e0 con 0x7ff038104d10 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.554+0000 7ff03e812640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff03810d6e0 con 0x7ff0381023e0 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03cdaf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038107640 0x7ff03819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03cdaf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038107640 0x7ff03819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41416/0 (socket says 172.21.15.49:41416) 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03cdaf640 1 -- 172.21.15.49:0/1213995428 learned_addr learned my addr 172.21.15.49:0/1213995428 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff037fff640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0381023e0 0x7ff038197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff0377fe640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff038104d10 0x7ff038198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03cdaf640 1 -- 172.21.15.49:0/1213995428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0381023e0 msgr2=0x7ff038197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03cdaf640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0381023e0 0x7ff038197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03cdaf640 1 -- 172.21.15.49:0/1213995428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff038104d10 msgr2=0x7ff038198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:47.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03cdaf640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff038104d10 0x7ff038198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:47.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03cdaf640 1 -- 172.21.15.49:0/1213995428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03819f840 con 0x7ff038107640 2024-03-26T17:59:47.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03cdaf640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038107640 0x7ff03819f1a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff030037040 tx=0x7ff030032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:47.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff0357fa640 1 -- 172.21.15.49:0/1213995428 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030041020 con 0x7ff038107640 2024-03-26T17:59:47.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff0357fa640 1 -- 172.21.15.49:0/1213995428 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff030034d90 con 0x7ff038107640 2024-03-26T17:59:47.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03e812640 1 -- 172.21.15.49:0/1213995428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03819fad0 con 0x7ff038107640 2024-03-26T17:59:47.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff0357fa640 1 -- 172.21.15.49:0/1213995428 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030039040 con 0x7ff038107640 2024-03-26T17:59:47.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.555+0000 7ff03e812640 1 -- 172.21.15.49:0/1213995428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03819ffc0 con 0x7ff038107640 2024-03-26T17:59:47.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.556+0000 7ff03e812640 1 -- 172.21.15.49:0/1213995428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0380fd840 con 0x7ff038107640 2024-03-26T17:59:47.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.557+0000 7ff0357fa640 1 -- 172.21.15.49:0/1213995428 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7ff030044020 con 0x7ff038107640 2024-03-26T17:59:47.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.558+0000 7ff0357fa640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0200a2f20 0x7ff0200a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:47.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.558+0000 7ff037fff640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0200a2f20 0x7ff0200a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:47.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.558+0000 7ff0357fa640 1 -- 172.21.15.49:0/1213995428 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff0300f12e0 con 0x7ff038107640 2024-03-26T17:59:47.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.559+0000 7ff037fff640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0200a2f20 0x7ff0200a53e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff038199100 tx=0x7ff028005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:47.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.562+0000 7ff0357fa640 1 -- 172.21.15.49:0/1213995428 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff03004b020 con 0x7ff038107640 2024-03-26T17:59:47.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1885542751' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-26T17:59:47.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T17:59:47.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3399640560' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T17:59:47.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T17:59:47.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T17:59:47.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:47.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:47.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:47 smithi115 ceph-mon[36174]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T17:59:47.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:47.759+0000 7ff03e812640 1 -- 172.21.15.49:0/1213995428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ff0380624c0 con 0x7ff038107640 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1885542751' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3399640560' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:47.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:47.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:47 smithi049 ceph-mon[32331]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T17:59:47.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1885542751' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-26T17:59:47.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T17:59:47.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3399640560' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T17:59:47.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T17:59:47.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T17:59:47.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:47.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:47.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:47.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:47 smithi171 ceph-mon[36033]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T17:59:48.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.226+0000 7ff0357fa640 1 -- 172.21.15.49:0/1213995428 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7ff0300be1c0 con 0x7ff038107640 2024-03-26T17:59:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 -- 172.21.15.49:0/1213995428 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0200a2f20 msgr2=0x7ff0200a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0200a2f20 0x7ff0200a53e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff038199100 tx=0x7ff028005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 -- 172.21.15.49:0/1213995428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038107640 msgr2=0x7ff03819f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038107640 0x7ff03819f1a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff030037040 tx=0x7ff030032f20 comp rx=0 tx=0).stop 2024-03-26T17:59:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 -- 172.21.15.49:0/1213995428 shutdown_connections 2024-03-26T17:59:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff0200a2f20 0x7ff0200a53e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff038107640 0x7ff03819f1a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff038104d10 0x7ff038198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 --2- 172.21.15.49:0/1213995428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0381023e0 0x7ff038197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 -- 172.21.15.49:0/1213995428 >> 172.21.15.49:0/1213995428 conn(0x7ff0380fbd50 msgr2=0x7ff038102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 -- 172.21.15.49:0/1213995428 shutdown_connections 2024-03-26T17:59:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.229+0000 7ff03e812640 1 -- 172.21.15.49:0/1213995428 wait complete. 2024-03-26T17:59:48.225 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-26T17:59:48.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.363+0000 7fc981d3b640 1 -- 172.21.15.49:0/2451233917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc97c104d10 msgr2=0x7fc97c107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:48.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.363+0000 7fc981d3b640 1 --2- 172.21.15.49:0/2451233917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc97c104d10 0x7fc97c107100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc970009900 tx=0x7fc97002bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:48.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.363+0000 7fc981d3b640 1 -- 172.21.15.49:0/2451233917 shutdown_connections 2024-03-26T17:59:48.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.363+0000 7fc981d3b640 1 --2- 172.21.15.49:0/2451233917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc97c107640 0x7fc97c109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:48.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.363+0000 7fc981d3b640 1 --2- 172.21.15.49:0/2451233917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc97c104d10 0x7fc97c107100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:48.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.363+0000 7fc981d3b640 1 --2- 172.21.15.49:0/2451233917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc97c1023e0 0x7fc97c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.363+0000 7fc981d3b640 1 -- 172.21.15.49:0/2451233917 >> 172.21.15.49:0/2451233917 conn(0x7fc97c0fbdf0 msgr2=0x7fc97c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.363+0000 7fc981d3b640 1 -- 172.21.15.49:0/2451233917 shutdown_connections 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.363+0000 7fc981d3b640 1 -- 172.21.15.49:0/2451233917 wait complete. 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.363+0000 7fc981d3b640 1 Processor -- start 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc981d3b640 1 -- start start 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc981d3b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc97c1023e0 0x7fc97c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc981d3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc97c104d10 0x7fc97c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc981d3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc97c107640 0x7fc97c19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc981d3b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c10d5b0 con 0x7fc97c104d10 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc981d3b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c10d430 con 0x7fc97c107640 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc981d3b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c10d730 con 0x7fc97c1023e0 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc97c104d10 0x7fc97c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:48.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc97c104d10 0x7fc97c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41434/0 (socket says 172.21.15.49:41434) 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97affd640 1 -- 172.21.15.49:0/2258516776 learned_addr learned my addr 172.21.15.49:0/2258516776 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97b7fe640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc97c1023e0 0x7fc97c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97bfff640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc97c107640 0x7fc97c19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97affd640 1 -- 172.21.15.49:0/2258516776 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc97c1023e0 msgr2=0x7fc97c197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97affd640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc97c1023e0 0x7fc97c197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97affd640 1 -- 172.21.15.49:0/2258516776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc97c107640 msgr2=0x7fc97c19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97affd640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc97c107640 0x7fc97c19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97affd640 1 -- 172.21.15.49:0/2258516776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c19f850 con 0x7fc97c104d10 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc97affd640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc97c104d10 0x7fc97c198180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc970000c00 tx=0x7fc970005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.364+0000 7fc978ff9640 1 -- 172.21.15.49:0/2258516776 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970043020 con 0x7fc97c104d10 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.365+0000 7fc981d3b640 1 -- 172.21.15.49:0/2258516776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97c19fae0 con 0x7fc97c104d10 2024-03-26T17:59:48.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.365+0000 7fc978ff9640 1 -- 172.21.15.49:0/2258516776 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9700054e0 con 0x7fc97c104d10 2024-03-26T17:59:48.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.365+0000 7fc978ff9640 1 -- 172.21.15.49:0/2258516776 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970003e60 con 0x7fc97c104d10 2024-03-26T17:59:48.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.365+0000 7fc981d3b640 1 -- 172.21.15.49:0/2258516776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97c1a0080 con 0x7fc97c104d10 2024-03-26T17:59:48.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.366+0000 7fc981d3b640 1 -- 172.21.15.49:0/2258516776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c0fd8e0 con 0x7fc97c104d10 2024-03-26T17:59:48.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.367+0000 7fc978ff9640 1 -- 172.21.15.49:0/2258516776 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fc970046020 con 0x7fc97c104d10 2024-03-26T17:59:48.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.367+0000 7fc978ff9640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc9640a2f20 0x7fc9640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:48.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.367+0000 7fc97b7fe640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc9640a2f20 0x7fc9640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:48.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.367+0000 7fc978ff9640 1 -- 172.21.15.49:0/2258516776 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fc9700f1600 con 0x7fc97c104d10 2024-03-26T17:59:48.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.368+0000 7fc97b7fe640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc9640a2f20 0x7fc9640a53e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc97c199110 tx=0x7fc96c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:48.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.372+0000 7fc978ff9640 1 -- 172.21.15.49:0/2258516776 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc97004d020 con 0x7fc97c104d10 2024-03-26T17:59:48.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:48 smithi049 ceph-mon[32331]: pgmap v286: 97 pgs: 20 creating+peering, 13 active+clean, 64 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:48.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:48 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3399640560' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T17:59:48.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:48 smithi049 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T17:59:48.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:48 smithi049 ceph-mon[32331]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T17:59:48.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:48 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:48.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:48 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1213995428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T17:59:48.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:48 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1213995428' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-26T17:59:48.503 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:48 smithi049 ceph-mon[32331]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T17:59:48.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:48.561+0000 7fc981d3b640 1 -- 172.21.15.49:0/2258516776 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc97c02d240 con 0x7fc97c104d10 2024-03-26T17:59:48.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:48 smithi115 ceph-mon[36174]: pgmap v286: 97 pgs: 20 creating+peering, 13 active+clean, 64 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:48.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:48 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3399640560' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T17:59:48.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:48 smithi115 ceph-mon[36174]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T17:59:48.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:48 smithi115 ceph-mon[36174]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T17:59:48.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:48 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:48.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:48 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1213995428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T17:59:48.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:48 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1213995428' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-26T17:59:48.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:48 smithi115 ceph-mon[36174]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T17:59:48.739 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 17:59:48 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-b[66478]: 2024-03-26T17:59:48.552+0000 7f53ae70c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-26T17:59:48.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:48 smithi171 ceph-mon[36033]: pgmap v286: 97 pgs: 20 creating+peering, 13 active+clean, 64 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:59:48.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:48 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3399640560' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T17:59:48.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:48 smithi171 ceph-mon[36033]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T17:59:48.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:48 smithi171 ceph-mon[36033]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T17:59:48.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:48 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T17:59:48.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:48 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1213995428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T17:59:48.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:48 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1213995428' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-26T17:59:48.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:48 smithi171 ceph-mon[36033]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T17:59:49.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:49 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] up:active 2024-03-26T17:59:49.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:49 smithi115 ceph-mon[36174]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:59:49.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:49 smithi115 ceph-mon[36174]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:59:49.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:49 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2258516776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T17:59:49.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:49 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] up:active 2024-03-26T17:59:49.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:49 smithi049 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:59:49.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:49 smithi049 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:59:49.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:49 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2258516776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T17:59:49.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:49 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] up:active 2024-03-26T17:59:49.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:49 smithi171 ceph-mon[36033]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:59:49.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:49 smithi171 ceph-mon[36033]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:59:49.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:49 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2258516776' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T17:59:50.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.452+0000 7fc978ff9640 1 -- 172.21.15.49:0/2258516776 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fc9700be400 con 0x7fc97c104d10 2024-03-26T17:59:50.435 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data_ec' created 2024-03-26T17:59:50.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 -- 172.21.15.49:0/2258516776 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc9640a2f20 msgr2=0x7fc9640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:50.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc9640a2f20 0x7fc9640a53e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc97c199110 tx=0x7fc96c00b410 comp rx=0 tx=0).stop 2024-03-26T17:59:50.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 -- 172.21.15.49:0/2258516776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc97c104d10 msgr2=0x7fc97c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:50.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc97c104d10 0x7fc97c198180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc970000c00 tx=0x7fc970005dd0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 -- 172.21.15.49:0/2258516776 shutdown_connections 2024-03-26T17:59:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc9640a2f20 0x7fc9640a53e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc97c107640 0x7fc97c19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc97c104d10 0x7fc97c198180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 --2- 172.21.15.49:0/2258516776 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc97c1023e0 0x7fc97c197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 -- 172.21.15.49:0/2258516776 >> 172.21.15.49:0/2258516776 conn(0x7fc97c0fbdf0 msgr2=0x7fc97c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.455+0000 7fc981d3b640 1 -- 172.21.15.49:0/2258516776 shutdown_connections 2024-03-26T17:59:50.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.456+0000 7fc981d3b640 1 -- 172.21.15.49:0/2258516776 wait complete. 2024-03-26T17:59:50.457 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-26T17:59:50.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 -- 172.21.15.49:0/3563201528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee81007c0 msgr2=0x7f7ee8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:50.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 --2- 172.21.15.49:0/3563201528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee81007c0 0x7f7ee8100be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009ca0 tx=0x7f7ed802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 -- 172.21.15.49:0/3563201528 shutdown_connections 2024-03-26T17:59:50.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 --2- 172.21.15.49:0/3563201528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ee8102370 0x7f7ee8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 --2- 172.21.15.49:0/3563201528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee81019b0 0x7f7ee8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 --2- 172.21.15.49:0/3563201528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee81007c0 0x7f7ee8100be0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 -- 172.21.15.49:0/3563201528 >> 172.21.15.49:0/3563201528 conn(0x7f7ee80fbd50 msgr2=0x7f7ee80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:50.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 -- 172.21.15.49:0/3563201528 shutdown_connections 2024-03-26T17:59:50.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 -- 172.21.15.49:0/3563201528 wait complete. 2024-03-26T17:59:50.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 Processor -- start 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.592+0000 7f7eee187640 1 -- start start 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7eee187640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ee81007c0 0x7f7ee8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7eee187640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee81019b0 0x7f7ee819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7eee187640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8102370 0x7f7ee819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7eee187640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee810c410 con 0x7f7ee81019b0 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7eee187640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee810c290 con 0x7f7ee8102370 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7eee187640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee810c590 con 0x7f7ee81007c0 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee81019b0 0x7f7ee819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee81019b0 0x7f7ee819c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58970/0 (socket says 172.21.15.49:58970) 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee6ffd640 1 -- 172.21.15.49:0/1117957557 learned_addr learned my addr 172.21.15.49:0/1117957557 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee7fff640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8102370 0x7f7ee819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:50.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee77fe640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ee81007c0 0x7f7ee8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee6ffd640 1 -- 172.21.15.49:0/1117957557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ee81007c0 msgr2=0x7f7ee8069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee6ffd640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ee81007c0 0x7f7ee8069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee6ffd640 1 -- 172.21.15.49:0/1117957557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8102370 msgr2=0x7f7ee819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee6ffd640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8102370 0x7f7ee819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee6ffd640 1 -- 172.21.15.49:0/1117957557 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee819f8c0 con 0x7f7ee81019b0 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee6ffd640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee81019b0 0x7f7ee819c990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00ea60 tx=0x7f7edc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee4ff9640 1 -- 172.21.15.49:0/1117957557 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc00cde0 con 0x7f7ee81019b0 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee4ff9640 1 -- 172.21.15.49:0/1117957557 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7edc0035f0 con 0x7f7ee81019b0 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7eee187640 1 -- 172.21.15.49:0/1117957557 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee819fb50 con 0x7f7ee81019b0 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7ee4ff9640 1 -- 172.21.15.49:0/1117957557 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc010670 con 0x7f7ee81019b0 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.593+0000 7f7eee187640 1 -- 172.21.15.49:0/1117957557 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee81a00f0 con 0x7f7ee81019b0 2024-03-26T17:59:50.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.594+0000 7f7eee187640 1 -- 172.21.15.49:0/1117957557 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee8107c00 con 0x7f7ee81019b0 2024-03-26T17:59:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.596+0000 7f7ee4ff9640 1 -- 172.21.15.49:0/1117957557 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f7edc0037a0 con 0x7f7ee81019b0 2024-03-26T17:59:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.597+0000 7f7ee4ff9640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7ed00a2fa0 0x7f7ed00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.597+0000 7f7ee77fe640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7ed00a2fa0 0x7f7ed00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.597+0000 7f7ee4ff9640 1 -- 172.21.15.49:0/1117957557 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f7edc0cf930 con 0x7f7ee81019b0 2024-03-26T17:59:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.597+0000 7f7ee77fe640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7ed00a2fa0 0x7f7ed00a5460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7ed802c700 tx=0x7f7ed8032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:50.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.601+0000 7f7ee4ff9640 1 -- 172.21.15.49:0/1117957557 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7edc09c5d0 con 0x7f7ee81019b0 2024-03-26T17:59:50.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:50 smithi115 ceph-mon[36174]: pgmap v289: 97 pgs: 20 creating+peering, 41 active+clean, 36 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-26T17:59:50.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:50 smithi115 ceph-mon[36174]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T17:59:50.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:50 smithi115 ceph-mon[36174]: Cluster is now healthy 2024-03-26T17:59:50.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:50 smithi115 ceph-mon[36174]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T17:59:50.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:50 smithi049 ceph-mon[32331]: pgmap v289: 97 pgs: 20 creating+peering, 41 active+clean, 36 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-26T17:59:50.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:50 smithi049 ceph-mon[32331]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T17:59:50.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:50 smithi049 ceph-mon[32331]: Cluster is now healthy 2024-03-26T17:59:50.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:50 smithi049 ceph-mon[32331]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T17:59:50.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:50.785+0000 7f7eee187640 1 -- 172.21.15.49:0/1117957557 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f7ee802d240 con 0x7f7ee81019b0 2024-03-26T17:59:50.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:50 smithi171 ceph-mon[36033]: pgmap v289: 97 pgs: 20 creating+peering, 41 active+clean, 36 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-26T17:59:50.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:50 smithi171 ceph-mon[36033]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T17:59:50.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:50 smithi171 ceph-mon[36033]: Cluster is now healthy 2024-03-26T17:59:50.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:50 smithi171 ceph-mon[36033]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T17:59:51.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.457+0000 7f7ee4ff9640 1 -- 172.21.15.49:0/1117957557 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7f7edc0a2070 con 0x7f7ee81019b0 2024-03-26T17:59:51.440 INFO:teuthology.orchestra.run.smithi049.stderr:set pool 4 allow_ec_overwrites to true 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 -- 172.21.15.49:0/1117957557 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7ed00a2fa0 msgr2=0x7f7ed00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7ed00a2fa0 0x7f7ed00a5460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7ed802c700 tx=0x7f7ed8032590 comp rx=0 tx=0).stop 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 -- 172.21.15.49:0/1117957557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee81019b0 msgr2=0x7f7ee819c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee81019b0 0x7f7ee819c990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00ea60 tx=0x7f7edc00ef20 comp rx=0 tx=0).stop 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 -- 172.21.15.49:0/1117957557 shutdown_connections 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7ed00a2fa0 0x7f7ed00a5460 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ee8102370 0x7f7ee819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ee81019b0 0x7f7ee819c990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 --2- 172.21.15.49:0/1117957557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ee81007c0 0x7f7ee8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 -- 172.21.15.49:0/1117957557 >> 172.21.15.49:0/1117957557 conn(0x7f7ee80fbd50 msgr2=0x7f7ee8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 -- 172.21.15.49:0/1117957557 shutdown_connections 2024-03-26T17:59:51.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.461+0000 7f7eee187640 1 -- 172.21.15.49:0/1117957557 wait complete. 2024-03-26T17:59:51.466 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-26T17:59:51.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 -- 172.21.15.49:0/2896732530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c0102370 msgr2=0x7f77c0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:51.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 --2- 172.21.15.49:0/2896732530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c0102370 0x7f77c0108c00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009ca0 tx=0x7f77b802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:51.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 -- 172.21.15.49:0/2896732530 shutdown_connections 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 --2- 172.21.15.49:0/2896732530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c01019b0 0x7f77c0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 --2- 172.21.15.49:0/2896732530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c0102370 0x7f77c0108c00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 --2- 172.21.15.49:0/2896732530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c01007c0 0x7f77c0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 -- 172.21.15.49:0/2896732530 >> 172.21.15.49:0/2896732530 conn(0x7f77c00fbd30 msgr2=0x7f77c00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 -- 172.21.15.49:0/2896732530 shutdown_connections 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 -- 172.21.15.49:0/2896732530 wait complete. 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 Processor -- start 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 -- start start 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c01007c0 0x7f77c00695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c01019b0 0x7f77c019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c0102370 0x7f77c019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:51.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010c4a0 con 0x7f77c01007c0 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010c320 con 0x7f77c01019b0 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c6de6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010c620 con 0x7f77c0102370 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c4b82640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c01007c0 0x7f77c00695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.601+0000 7f77c4b82640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c01007c0 0x7f77c00695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58990/0 (socket says 172.21.15.49:58990) 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77c4b82640 1 -- 172.21.15.49:0/1762128671 learned_addr learned my addr 172.21.15.49:0/1762128671 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77bffff640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c01019b0 0x7f77c019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77c4b82640 1 -- 172.21.15.49:0/1762128671 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c0102370 msgr2=0x7f77c019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77c4b82640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c0102370 0x7f77c019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77c4b82640 1 -- 172.21.15.49:0/1762128671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c01019b0 msgr2=0x7f77c019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77c4b82640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c01019b0 0x7f77c019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77c4b82640 1 -- 172.21.15.49:0/1762128671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c019f7e0 con 0x7f77c01007c0 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77c4b82640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c01007c0 0x7f77c00695b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f77a800e9c0 tx=0x7f77a800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77bdffb640 1 -- 172.21.15.49:0/1762128671 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a800ccc0 con 0x7f77c01007c0 2024-03-26T17:59:51.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77bdffb640 1 -- 172.21.15.49:0/1762128671 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a80035f0 con 0x7f77c01007c0 2024-03-26T17:59:51.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77bdffb640 1 -- 172.21.15.49:0/1762128671 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a800ce60 con 0x7f77c01007c0 2024-03-26T17:59:51.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77c6de6640 1 -- 172.21.15.49:0/1762128671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c019fad0 con 0x7f77c01007c0 2024-03-26T17:59:51.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.602+0000 7f77c6de6640 1 -- 172.21.15.49:0/1762128671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c01a0070 con 0x7f77c01007c0 2024-03-26T17:59:51.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.603+0000 7f77c6de6640 1 -- 172.21.15.49:0/1762128671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c0107a80 con 0x7f77c01007c0 2024-03-26T17:59:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.604+0000 7f77bdffb640 1 -- 172.21.15.49:0/1762128671 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f77a80037a0 con 0x7f77c01007c0 2024-03-26T17:59:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.605+0000 7f77bdffb640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f77ac0a2fa0 0x7f77ac0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.605+0000 7f77bffff640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f77ac0a2fa0 0x7f77ac0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.605+0000 7f77bdffb640 1 -- 172.21.15.49:0/1762128671 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f77a80cf9b0 con 0x7f77c01007c0 2024-03-26T17:59:51.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.606+0000 7f77bffff640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f77ac0a2fa0 0x7f77ac0a5460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f77c0069320 tx=0x7f77b0009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:51.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.610+0000 7f77bdffb640 1 -- 172.21.15.49:0/1762128671 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f77a809c560 con 0x7f77c01007c0 2024-03-26T17:59:51.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:51 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2258516776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T17:59:51.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:51 smithi115 ceph-mon[36174]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T17:59:51.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:51 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1117957557' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-26T17:59:51.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:51 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2258516776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T17:59:51.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:51 smithi049 ceph-mon[32331]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T17:59:51.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:51 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1117957557' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-26T17:59:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:51.835+0000 7f77c6de6640 1 -- 172.21.15.49:0/1762128671 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f77c0062560 con 0x7f77c01007c0 2024-03-26T17:59:51.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:51 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2258516776' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T17:59:51.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:51 smithi171 ceph-mon[36033]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T17:59:51.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:51 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1117957557' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-26T17:59:52.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.464+0000 7f77bdffb640 1 -- 172.21.15.49:0/1762128671 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f77a809be60 con 0x7f77c01007c0 2024-03-26T17:59:52.447 INFO:teuthology.orchestra.run.smithi049.stderr:added data pool 4 to fsmap 2024-03-26T17:59:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 -- 172.21.15.49:0/1762128671 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f77ac0a2fa0 msgr2=0x7f77ac0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f77ac0a2fa0 0x7f77ac0a5460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f77c0069320 tx=0x7f77b0009110 comp rx=0 tx=0).stop 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 -- 172.21.15.49:0/1762128671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c01007c0 msgr2=0x7f77c00695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c01007c0 0x7f77c00695b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f77a800e9c0 tx=0x7f77a800edc0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 -- 172.21.15.49:0/1762128671 shutdown_connections 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f77ac0a2fa0 0x7f77ac0a5460 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c0102370 0x7f77c019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77c01019b0 0x7f77c019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 --2- 172.21.15.49:0/1762128671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f77c01007c0 0x7f77c00695b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 -- 172.21.15.49:0/1762128671 >> 172.21.15.49:0/1762128671 conn(0x7f77c00fbd30 msgr2=0x7f77c0104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 -- 172.21.15.49:0/1762128671 shutdown_connections 2024-03-26T17:59:52.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.467+0000 7f77c6de6640 1 -- 172.21.15.49:0/1762128671 wait complete. 2024-03-26T17:59:52.463 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:52.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.595+0000 7fbde4281640 1 -- 172.21.15.49:0/3404569693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbddc100860 msgr2=0x7fbddc100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:52.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.595+0000 7fbde4281640 1 --2- 172.21.15.49:0/3404569693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbddc100860 0x7fbddc100c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009900 tx=0x7fbdd802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.595+0000 7fbde4281640 1 -- 172.21.15.49:0/3404569693 shutdown_connections 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.595+0000 7fbde4281640 1 --2- 172.21.15.49:0/3404569693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbddc102410 0x7fbddc108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.595+0000 7fbde4281640 1 --2- 172.21.15.49:0/3404569693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbddc101a50 0x7fbddc101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.595+0000 7fbde4281640 1 --2- 172.21.15.49:0/3404569693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbddc100860 0x7fbddc100c80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.595+0000 7fbde4281640 1 -- 172.21.15.49:0/3404569693 >> 172.21.15.49:0/3404569693 conn(0x7fbddc0fbdf0 msgr2=0x7fbddc0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.595+0000 7fbde4281640 1 -- 172.21.15.49:0/3404569693 shutdown_connections 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.595+0000 7fbde4281640 1 -- 172.21.15.49:0/3404569693 wait complete. 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde4281640 1 Processor -- start 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde4281640 1 -- start start 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde4281640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbddc100860 0x7fbddc197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde4281640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbddc101a50 0x7fbddc198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:52.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde4281640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbddc102410 0x7fbddc19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde4281640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc10d590 con 0x7fbddc102410 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde4281640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc10d410 con 0x7fbddc101a50 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde4281640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc10d710 con 0x7fbddc100860 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde281e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbddc102410 0x7fbddc19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde281e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbddc102410 0x7fbddc19f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59010/0 (socket says 172.21.15.49:59010) 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde281e640 1 -- 172.21.15.49:0/1653278485 learned_addr learned my addr 172.21.15.49:0/1653278485 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde181c640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbddc101a50 0x7fbddc198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde201d640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbddc100860 0x7fbddc197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde281e640 1 -- 172.21.15.49:0/1653278485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbddc100860 msgr2=0x7fbddc197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde281e640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbddc100860 0x7fbddc197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde281e640 1 -- 172.21.15.49:0/1653278485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbddc101a50 msgr2=0x7fbddc198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:52.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde281e640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbddc101a50 0x7fbddc198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde281e640 1 -- 172.21.15.49:0/1653278485 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbddc19f830 con 0x7fbddc102410 2024-03-26T17:59:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde201d640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbddc100860 0x7fbddc197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.596+0000 7fbde181c640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbddc101a50 0x7fbddc198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.597+0000 7fbde281e640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbddc102410 0x7fbddc19f190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009fd0 tx=0x7fbdd400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.597+0000 7fbdd37fe640 1 -- 172.21.15.49:0/1653278485 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4017070 con 0x7fbddc102410 2024-03-26T17:59:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.597+0000 7fbdd37fe640 1 -- 172.21.15.49:0/1653278485 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd4003e50 con 0x7fbddc102410 2024-03-26T17:59:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.597+0000 7fbde4281640 1 -- 172.21.15.49:0/1653278485 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbddc19fb20 con 0x7fbddc102410 2024-03-26T17:59:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.597+0000 7fbdd37fe640 1 -- 172.21.15.49:0/1653278485 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd40106c0 con 0x7fbddc102410 2024-03-26T17:59:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.597+0000 7fbde4281640 1 -- 172.21.15.49:0/1653278485 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbddc1a0070 con 0x7fbddc102410 2024-03-26T17:59:52.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.598+0000 7fbde4281640 1 -- 172.21.15.49:0/1653278485 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbddc107b80 con 0x7fbddc102410 2024-03-26T17:59:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.599+0000 7fbdd37fe640 1 -- 172.21.15.49:0/1653278485 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fbdd4003970 con 0x7fbddc102410 2024-03-26T17:59:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.599+0000 7fbdd37fe640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbdc80a2fa0 0x7fbdc80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.600+0000 7fbdd37fe640 1 -- 172.21.15.49:0/1653278485 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbdd40cf400 con 0x7fbddc102410 2024-03-26T17:59:52.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.600+0000 7fbde201d640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbdc80a2fa0 0x7fbdc80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:52.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.600+0000 7fbde201d640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbdc80a2fa0 0x7fbdc80a5460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8006010 tx=0x7fbdd8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:52.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.604+0000 7fbdd37fe640 1 -- 172.21.15.49:0/1653278485 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbdd409bff0 con 0x7fbddc102410 2024-03-26T17:59:52.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:52 smithi115 ceph-mon[36174]: pgmap v292: 129 pgs: 23 creating+peering, 77 active+clean, 29 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-03-26T17:59:52.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:52 smithi115 ceph-mon[36174]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T17:59:52.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:52 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1117957557' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-26T17:59:52.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:52 smithi115 ceph-mon[36174]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T17:59:52.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:52 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1762128671' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-26T17:59:52.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:52 smithi049 ceph-mon[32331]: pgmap v292: 129 pgs: 23 creating+peering, 77 active+clean, 29 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-03-26T17:59:52.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:52 smithi049 ceph-mon[32331]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T17:59:52.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:52 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1117957557' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-26T17:59:52.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:52 smithi049 ceph-mon[32331]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T17:59:52.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:52 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1762128671' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-26T17:59:52.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.832+0000 7fbde4281640 1 -- 172.21.15.49:0/1653278485 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbddc02d240 con 0x7fbddc102410 2024-03-26T17:59:52.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.833+0000 7fbdd37fe640 1 -- 172.21.15.49:0/1653278485 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11809 (secure 0 0 0) 0x7fbdd409b8f0 con 0x7fbddc102410 2024-03-26T17:59:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:52.815 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-26T17:59:46.796381+0000","modified":"2024-03-26T17:59:51.836514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-03-26T17:59:52.816 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-03-26T17:59:52.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 -- 172.21.15.49:0/1653278485 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbdc80a2fa0 msgr2=0x7fbdc80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbdc80a2fa0 0x7fbdc80a5460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8006010 tx=0x7fbdd8000bc0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 -- 172.21.15.49:0/1653278485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbddc102410 msgr2=0x7fbddc19f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbddc102410 0x7fbddc19f190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009fd0 tx=0x7fbdd400b260 comp rx=0 tx=0).stop 2024-03-26T17:59:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 -- 172.21.15.49:0/1653278485 shutdown_connections 2024-03-26T17:59:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbdc80a2fa0 0x7fbdc80a5460 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbddc102410 0x7fbddc19f190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbddc101a50 0x7fbddc198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 --2- 172.21.15.49:0/1653278485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbddc100860 0x7fbddc197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 -- 172.21.15.49:0/1653278485 >> 172.21.15.49:0/1653278485 conn(0x7fbddc0fbdf0 msgr2=0x7fbddc104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:52.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 -- 172.21.15.49:0/1653278485 shutdown_connections 2024-03-26T17:59:52.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.835+0000 7fbde4281640 1 -- 172.21.15.49:0/1653278485 wait complete. 2024-03-26T17:59:52.829 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:59:52.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:52 smithi171 ceph-mon[36033]: pgmap v292: 129 pgs: 23 creating+peering, 77 active+clean, 29 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-03-26T17:59:52.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:52 smithi171 ceph-mon[36033]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T17:59:52.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:52 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1117957557' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-26T17:59:52.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:52 smithi171 ceph-mon[36033]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T17:59:52.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:52 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1762128671' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-26T17:59:52.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.954+0000 7fa0382ff640 1 -- 172.21.15.49:0/3435915406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0301023e0 msgr2=0x7fa0301047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:52.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.954+0000 7fa0382ff640 1 --2- 172.21.15.49:0/3435915406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0301023e0 0x7fa0301047d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009900 tx=0x7fa02c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:52.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.954+0000 7fa0382ff640 1 -- 172.21.15.49:0/3435915406 shutdown_connections 2024-03-26T17:59:52.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.954+0000 7fa0382ff640 1 --2- 172.21.15.49:0/3435915406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa030104d10 0x7fa030107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.954+0000 7fa0382ff640 1 --2- 172.21.15.49:0/3435915406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa030107640 0x7fa030109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.954+0000 7fa0382ff640 1 --2- 172.21.15.49:0/3435915406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0301023e0 0x7fa0301047d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.954+0000 7fa0382ff640 1 -- 172.21.15.49:0/3435915406 >> 172.21.15.49:0/3435915406 conn(0x7fa0300fbdf0 msgr2=0x7fa0300fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.954+0000 7fa0382ff640 1 -- 172.21.15.49:0/3435915406 shutdown_connections 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.954+0000 7fa0382ff640 1 -- 172.21.15.49:0/3435915406 wait complete. 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0382ff640 1 Processor -- start 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0382ff640 1 -- start start 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0382ff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0301023e0 0x7fa030197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0382ff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa030104d10 0x7fa0301981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0382ff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa030107640 0x7fa03019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0382ff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa03010d640 con 0x7fa0301023e0 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0382ff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa03010d4c0 con 0x7fa030104d10 2024-03-26T17:59:52.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0382ff640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa03010d7c0 con 0x7fa030107640 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03609b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0301023e0 0x7fa030197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03609b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0301023e0 0x7fa030197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59036/0 (socket says 172.21.15.49:59036) 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03609b640 1 -- 172.21.15.49:0/1781241441 learned_addr learned my addr 172.21.15.49:0/1781241441 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03589a640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa030104d10 0x7fa0301981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03609b640 1 -- 172.21.15.49:0/1781241441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa030107640 msgr2=0x7fa03019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03609b640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa030107640 0x7fa03019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03609b640 1 -- 172.21.15.49:0/1781241441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa030104d10 msgr2=0x7fa0301981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03609b640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa030104d10 0x7fa0301981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03609b640 1 -- 172.21.15.49:0/1781241441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03019f8d0 con 0x7fa0301023e0 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03589a640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa030104d10 0x7fa0301981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa03609b640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0301023e0 0x7fa030197c90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa02c005ab0 tx=0x7fa02c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:52.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0277fe640 1 -- 172.21.15.49:0/1781241441 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c043020 con 0x7fa0301023e0 2024-03-26T17:59:52.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0277fe640 1 -- 172.21.15.49:0/1781241441 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa02c0054e0 con 0x7fa0301023e0 2024-03-26T17:59:52.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.955+0000 7fa0277fe640 1 -- 172.21.15.49:0/1781241441 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c003ce0 con 0x7fa0301023e0 2024-03-26T17:59:52.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.956+0000 7fa0382ff640 1 -- 172.21.15.49:0/1781241441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03019fb60 con 0x7fa0301023e0 2024-03-26T17:59:52.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.956+0000 7fa0382ff640 1 -- 172.21.15.49:0/1781241441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0301a0100 con 0x7fa0301023e0 2024-03-26T17:59:52.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.956+0000 7fa0382ff640 1 -- 172.21.15.49:0/1781241441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0300fd8e0 con 0x7fa0301023e0 2024-03-26T17:59:52.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.957+0000 7fa0277fe640 1 -- 172.21.15.49:0/1781241441 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fa02c046020 con 0x7fa0301023e0 2024-03-26T17:59:52.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.958+0000 7fa0277fe640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa01c0a2fa0 0x7fa01c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:52.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.958+0000 7fa03589a640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa01c0a2fa0 0x7fa01c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:52.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.958+0000 7fa0277fe640 1 -- 172.21.15.49:0/1781241441 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa02c0f18c0 con 0x7fa0301023e0 2024-03-26T17:59:52.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.959+0000 7fa03589a640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa01c0a2fa0 0x7fa01c0a5460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa030199190 tx=0x7fa020008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:52.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:52.962+0000 7fa0277fe640 1 -- 172.21.15.49:0/1781241441 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa02c04d020 con 0x7fa0301023e0 2024-03-26T17:59:53.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.147+0000 7fa0382ff640 1 -- 172.21.15.49:0/1781241441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa030062560 con 0x7fa0301023e0 2024-03-26T17:59:53.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.148+0000 7fa0277fe640 1 -- 172.21.15.49:0/1781241441 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24136 (secure 0 0 0) 0x7fa02c0be4b0 con 0x7fa0301023e0 2024-03-26T17:59:53.131 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:53.131 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:51.836345+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:59:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 -- 172.21.15.49:0/1781241441 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa01c0a2fa0 msgr2=0x7fa01c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa01c0a2fa0 0x7fa01c0a5460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa030199190 tx=0x7fa020008040 comp rx=0 tx=0).stop 2024-03-26T17:59:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 -- 172.21.15.49:0/1781241441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0301023e0 msgr2=0x7fa030197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0301023e0 0x7fa030197c90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa02c005ab0 tx=0x7fa02c02c6c0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 -- 172.21.15.49:0/1781241441 shutdown_connections 2024-03-26T17:59:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa01c0a2fa0 0x7fa01c0a5460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa030107640 0x7fa03019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa030104d10 0x7fa0301981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 --2- 172.21.15.49:0/1781241441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0301023e0 0x7fa030197c90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 -- 172.21.15.49:0/1781241441 >> 172.21.15.49:0/1781241441 conn(0x7fa0300fbdf0 msgr2=0x7fa030102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 -- 172.21.15.49:0/1781241441 shutdown_connections 2024-03-26T17:59:53.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.150+0000 7fa0382ff640 1 -- 172.21.15.49:0/1781241441 wait complete. 2024-03-26T17:59:53.145 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:59:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.270+0000 7efc6d75b640 1 -- 172.21.15.49:0/806122930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc68102400 msgr2=0x7efc681047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.270+0000 7efc6d75b640 1 --2- 172.21.15.49:0/806122930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc68102400 0x7efc681047f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc60009900 tx=0x7efc6002bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.270+0000 7efc6d75b640 1 -- 172.21.15.49:0/806122930 shutdown_connections 2024-03-26T17:59:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.270+0000 7efc6d75b640 1 --2- 172.21.15.49:0/806122930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68104d30 0x7efc68107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.270+0000 7efc6d75b640 1 --2- 172.21.15.49:0/806122930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc68107660 0x7efc68109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.270+0000 7efc6d75b640 1 --2- 172.21.15.49:0/806122930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc68102400 0x7efc681047f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.270+0000 7efc6d75b640 1 -- 172.21.15.49:0/806122930 >> 172.21.15.49:0/806122930 conn(0x7efc680fbd50 msgr2=0x7efc680fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.270+0000 7efc6d75b640 1 -- 172.21.15.49:0/806122930 shutdown_connections 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.270+0000 7efc6d75b640 1 -- 172.21.15.49:0/806122930 wait complete. 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc6d75b640 1 Processor -- start 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc6d75b640 1 -- start start 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc6d75b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc68102400 0x7efc68197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc6d75b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68104d30 0x7efc68198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc6d75b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc68107660 0x7efc6819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc6d75b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810d580 con 0x7efc68107660 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc6d75b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810d400 con 0x7efc68102400 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc6d75b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efc6810d700 con 0x7efc68104d30 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc68107660 0x7efc6819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc68107660 0x7efc6819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59068/0 (socket says 172.21.15.49:59068) 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc66ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc68102400 0x7efc68197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc677fe640 1 -- 172.21.15.49:0/3592696576 learned_addr learned my addr 172.21.15.49:0/3592696576 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.271+0000 7efc667fc640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68104d30 0x7efc68198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc677fe640 1 -- 172.21.15.49:0/3592696576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68104d30 msgr2=0x7efc68198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc677fe640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68104d30 0x7efc68198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc677fe640 1 -- 172.21.15.49:0/3592696576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc68102400 msgr2=0x7efc68197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc677fe640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc68102400 0x7efc68197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc677fe640 1 -- 172.21.15.49:0/3592696576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6819f850 con 0x7efc68107660 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc66ffd640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc68102400 0x7efc68197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc677fe640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc68107660 0x7efc6819f1b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efc5800eb10 tx=0x7efc5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc4ffff640 1 -- 172.21.15.49:0/3592696576 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004b20 con 0x7efc68107660 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc4ffff640 1 -- 172.21.15.49:0/3592696576 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc58004cc0 con 0x7efc68107660 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc6d75b640 1 -- 172.21.15.49:0/3592696576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6819fb40 con 0x7efc68107660 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc4ffff640 1 -- 172.21.15.49:0/3592696576 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58010760 con 0x7efc68107660 2024-03-26T17:59:53.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.272+0000 7efc6d75b640 1 -- 172.21.15.49:0/3592696576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc681a0090 con 0x7efc68107660 2024-03-26T17:59:53.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.273+0000 7efc6d75b640 1 -- 172.21.15.49:0/3592696576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc680fd840 con 0x7efc68107660 2024-03-26T17:59:53.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.274+0000 7efc4ffff640 1 -- 172.21.15.49:0/3592696576 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7efc58005000 con 0x7efc68107660 2024-03-26T17:59:53.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.275+0000 7efc4ffff640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc500a2fa0 0x7efc500a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.275+0000 7efc66ffd640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc500a2fa0 0x7efc500a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.275+0000 7efc4ffff640 1 -- 172.21.15.49:0/3592696576 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7efc580cfe50 con 0x7efc68107660 2024-03-26T17:59:53.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.275+0000 7efc66ffd640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc500a2fa0 0x7efc500a5460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc68199110 tx=0x7efc60005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.278+0000 7efc4ffff640 1 -- 172.21.15.49:0/3592696576 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efc58053740 con 0x7efc68107660 2024-03-26T17:59:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.465+0000 7efc6d75b640 1 -- 172.21.15.49:0/3592696576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc6802d490 con 0x7efc68107660 2024-03-26T17:59:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.468+0000 7efc4ffff640 1 -- 172.21.15.49:0/3592696576 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24136 (secure 0 0 0) 0x7efc58096e20 con 0x7efc68107660 2024-03-26T17:59:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:53.451 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:51.836345+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:59:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 -- 172.21.15.49:0/3592696576 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc500a2fa0 msgr2=0x7efc500a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc500a2fa0 0x7efc500a5460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc68199110 tx=0x7efc60005fd0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 -- 172.21.15.49:0/3592696576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc68107660 msgr2=0x7efc6819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc68107660 0x7efc6819f1b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efc5800eb10 tx=0x7efc5800c2d0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 -- 172.21.15.49:0/3592696576 shutdown_connections 2024-03-26T17:59:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc500a2fa0 0x7efc500a5460 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc68104d30 0x7efc68198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc68107660 0x7efc6819f1b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 --2- 172.21.15.49:0/3592696576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc68102400 0x7efc68197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 -- 172.21.15.49:0/3592696576 >> 172.21.15.49:0/3592696576 conn(0x7efc680fbd50 msgr2=0x7efc68102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 -- 172.21.15.49:0/3592696576 shutdown_connections 2024-03-26T17:59:53.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.470+0000 7efc6d75b640 1 -- 172.21.15.49:0/3592696576 wait complete. 2024-03-26T17:59:53.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.605+0000 7f5234797640 1 -- 172.21.15.49:0/1747665294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f522c107640 msgr2=0x7f522c109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.605+0000 7f5234797640 1 --2- 172.21.15.49:0/1747665294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f522c107640 0x7f522c109a90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5224009ca0 tx=0x7f522402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.605+0000 7f5234797640 1 -- 172.21.15.49:0/1747665294 shutdown_connections 2024-03-26T17:59:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.605+0000 7f5234797640 1 --2- 172.21.15.49:0/1747665294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f522c104d10 0x7f522c107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.605+0000 7f5234797640 1 --2- 172.21.15.49:0/1747665294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f522c107640 0x7f522c109a90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.605+0000 7f5234797640 1 --2- 172.21.15.49:0/1747665294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f522c1023e0 0x7f522c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.605+0000 7f5234797640 1 -- 172.21.15.49:0/1747665294 >> 172.21.15.49:0/1747665294 conn(0x7f522c0fbdf0 msgr2=0x7f522c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.605+0000 7f5234797640 1 -- 172.21.15.49:0/1747665294 shutdown_connections 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.605+0000 7f5234797640 1 -- 172.21.15.49:0/1747665294 wait complete. 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5234797640 1 Processor -- start 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5234797640 1 -- start start 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5234797640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f522c1023e0 0x7f522c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5234797640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f522c104d10 0x7f522c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5234797640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f522c107640 0x7f522c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5234797640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f522c10d570 con 0x7f522c1023e0 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5234797640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f522c10d3f0 con 0x7f522c104d10 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5234797640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f522c10d6f0 con 0x7f522c107640 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5232533640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f522c1023e0 0x7f522c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5232533640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f522c1023e0 0x7f522c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59086/0 (socket says 172.21.15.49:59086) 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5232533640 1 -- 172.21.15.49:0/12657815 learned_addr learned my addr 172.21.15.49:0/12657815 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.606+0000 7f5231d32640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f522c104d10 0x7f522c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f5232533640 1 -- 172.21.15.49:0/12657815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f522c107640 msgr2=0x7f522c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f5232533640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f522c107640 0x7f522c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f5232533640 1 -- 172.21.15.49:0/12657815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f522c104d10 msgr2=0x7f522c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f5232533640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f522c104d10 0x7f522c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f5232533640 1 -- 172.21.15.49:0/12657815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c19f730 con 0x7f522c1023e0 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f5232533640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f522c1023e0 0x7f522c197b10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f522800e9f0 tx=0x7f522800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f52237fe640 1 -- 172.21.15.49:0/12657815 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522800ccc0 con 0x7f522c1023e0 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f52237fe640 1 -- 172.21.15.49:0/12657815 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5228010910 con 0x7f522c1023e0 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f5234797640 1 -- 172.21.15.49:0/12657815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522c19fa20 con 0x7f522c1023e0 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f52237fe640 1 -- 172.21.15.49:0/12657815 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52280188a0 con 0x7f522c1023e0 2024-03-26T17:59:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.607+0000 7f5234797640 1 -- 172.21.15.49:0/12657815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522c19ffc0 con 0x7f522c1023e0 2024-03-26T17:59:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.608+0000 7f5234797640 1 -- 172.21.15.49:0/12657815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522c0fd8e0 con 0x7f522c1023e0 2024-03-26T17:59:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.609+0000 7f52237fe640 1 -- 172.21.15.49:0/12657815 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f5228010ac0 con 0x7f522c1023e0 2024-03-26T17:59:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.610+0000 7f52237fe640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52180a2fa0 0x7f52180a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.610+0000 7f52237fe640 1 -- 172.21.15.49:0/12657815 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f52280cff00 con 0x7f522c1023e0 2024-03-26T17:59:53.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.610+0000 7f5231d32640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52180a2fa0 0x7f52180a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.610+0000 7f5231d32640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52180a2fa0 0x7f52180a5460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f522c199020 tx=0x7f521c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.613+0000 7f52237fe640 1 -- 172.21.15.49:0/12657815 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f522809caf0 con 0x7f522c1023e0 2024-03-26T17:59:53.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:53 smithi115 ceph-mon[36174]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T17:59:53.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:53 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1762128671' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-26T17:59:53.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:53 smithi115 ceph-mon[36174]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:59:53.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:53 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1653278485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:53.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:53 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1781241441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:53.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:53 smithi049 ceph-mon[32331]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T17:59:53.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:53 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1762128671' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-26T17:59:53.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:53 smithi049 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:59:53.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:53 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1653278485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:53.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:53 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1781241441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:53.758 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:53 smithi171 ceph-mon[36033]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T17:59:53.758 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:53 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1762128671' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-26T17:59:53.758 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:53 smithi171 ceph-mon[36033]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:59:53.758 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:53 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1653278485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:53.758 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:53 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1781241441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:53.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.835+0000 7f5234797640 1 -- 172.21.15.49:0/12657815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f522c02d240 con 0x7f522c1023e0 2024-03-26T17:59:53.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.836+0000 7f52237fe640 1 -- 172.21.15.49:0/12657815 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11809 (secure 0 0 0) 0x7f522809c3f0 con 0x7f522c1023e0 2024-03-26T17:59:53.819 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:53.819 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-26T17:59:46.796381+0000","modified":"2024-03-26T17:59:51.836514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24533},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24533,"qdb_cluster":[24533]},"id":1}]} 2024-03-26T17:59:53.819 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2024-03-26T17:59:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 -- 172.21.15.49:0/12657815 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52180a2fa0 msgr2=0x7f52180a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52180a2fa0 0x7f52180a5460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f522c199020 tx=0x7f521c00a2d0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 -- 172.21.15.49:0/12657815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f522c1023e0 msgr2=0x7f522c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f522c1023e0 0x7f522c197b10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f522800e9f0 tx=0x7f522800edf0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 -- 172.21.15.49:0/12657815 shutdown_connections 2024-03-26T17:59:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52180a2fa0 0x7f52180a5460 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f522c107640 0x7f522c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f522c104d10 0x7f522c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 --2- 172.21.15.49:0/12657815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f522c1023e0 0x7f522c197b10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 -- 172.21.15.49:0/12657815 >> 172.21.15.49:0/12657815 conn(0x7f522c0fbdf0 msgr2=0x7f522c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 -- 172.21.15.49:0/12657815 shutdown_connections 2024-03-26T17:59:53.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.838+0000 7f5234797640 1 -- 172.21.15.49:0/12657815 wait complete. 2024-03-26T17:59:53.832 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.958+0000 7f41fdcf1640 1 -- 172.21.15.49:0/1892442784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41f8102ae0 msgr2=0x7f41f8102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.958+0000 7f41fdcf1640 1 --2- 172.21.15.49:0/1892442784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41f8102ae0 0x7f41f8102f60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009900 tx=0x7f41ec02bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.958+0000 7f41fdcf1640 1 -- 172.21.15.49:0/1892442784 shutdown_connections 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.958+0000 7f41fdcf1640 1 --2- 172.21.15.49:0/1892442784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f81034a0 0x7f41f8109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.958+0000 7f41fdcf1640 1 --2- 172.21.15.49:0/1892442784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41f8102ae0 0x7f41f8102f60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.958+0000 7f41fdcf1640 1 --2- 172.21.15.49:0/1892442784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41f81018f0 0x7f41f8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.958+0000 7f41fdcf1640 1 -- 172.21.15.49:0/1892442784 >> 172.21.15.49:0/1892442784 conn(0x7f41f80fcea0 msgr2=0x7f41f80ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.958+0000 7f41fdcf1640 1 -- 172.21.15.49:0/1892442784 shutdown_connections 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.958+0000 7f41fdcf1640 1 -- 172.21.15.49:0/1892442784 wait complete. 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.958+0000 7f41fdcf1640 1 Processor -- start 2024-03-26T17:59:53.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41fdcf1640 1 -- start start 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41fdcf1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41f81018f0 0x7f41f8197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41fdcf1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f8102ae0 0x7f41f8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41fdcf1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41f81034a0 0x7f41f819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41fdcf1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f41f810d570 con 0x7f41f81034a0 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41fdcf1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f41f810d3f0 con 0x7f41f8102ae0 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41fdcf1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f41f810d6f0 con 0x7f41f81018f0 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41f81034a0 0x7f41f819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f77fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41f81018f0 0x7f41f8197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f77fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41f81018f0 0x7f41f8197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:57516/0 (socket says 172.21.15.49:57516) 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41f81034a0 0x7f41f819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59104/0 (socket says 172.21.15.49:59104) 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f77fe640 1 -- 172.21.15.49:0/61793913 learned_addr learned my addr 172.21.15.49:0/61793913 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f6ffd640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f8102ae0 0x7f41f8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f7fff640 1 -- 172.21.15.49:0/61793913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41f81018f0 msgr2=0x7f41f8197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f7fff640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41f81018f0 0x7f41f8197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f7fff640 1 -- 172.21.15.49:0/61793913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f8102ae0 msgr2=0x7f41f8198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f7fff640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f8102ae0 0x7f41f8198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f7fff640 1 -- 172.21.15.49:0/61793913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f819f850 con 0x7f41f81034a0 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f6ffd640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f8102ae0 0x7f41f8198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f77fe640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41f81018f0 0x7f41f8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.959+0000 7f41f7fff640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41f81034a0 0x7f41f819f1b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009fd0 tx=0x7f41f000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.960+0000 7f41f4ff9640 1 -- 172.21.15.49:0/61793913 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f000ce20 con 0x7f41f81034a0 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.960+0000 7f41f4ff9640 1 -- 172.21.15.49:0/61793913 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f0005020 con 0x7f41f81034a0 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.960+0000 7f41fdcf1640 1 -- 172.21.15.49:0/61793913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f819fb40 con 0x7f41f81034a0 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.960+0000 7f41f4ff9640 1 -- 172.21.15.49:0/61793913 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0010720 con 0x7f41f81034a0 2024-03-26T17:59:53.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.960+0000 7f41fdcf1640 1 -- 172.21.15.49:0/61793913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f81a0040 con 0x7f41f81034a0 2024-03-26T17:59:53.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.961+0000 7f41fdcf1640 1 -- 172.21.15.49:0/61793913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f8108c10 con 0x7f41f81034a0 2024-03-26T17:59:53.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.962+0000 7f41f4ff9640 1 -- 172.21.15.49:0/61793913 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f41f0005540 con 0x7f41f81034a0 2024-03-26T17:59:53.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.962+0000 7f41f4ff9640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f41e00a2f20 0x7f41e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.962+0000 7f41f77fe640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f41e00a2f20 0x7f41e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.962+0000 7f41f4ff9640 1 -- 172.21.15.49:0/61793913 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f41f00cfd60 con 0x7f41f81034a0 2024-03-26T17:59:53.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.963+0000 7f41f77fe640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f41e00a2f20 0x7f41e00a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f41f8199110 tx=0x7f41e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:53.966+0000 7f41f4ff9640 1 -- 172.21.15.49:0/61793913 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f41f009c950 con 0x7f41f81034a0 2024-03-26T17:59:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.149+0000 7f41fdcf1640 1 -- 172.21.15.49:0/61793913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41f802d410 con 0x7f41f81034a0 2024-03-26T17:59:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.151+0000 7f41f4ff9640 1 -- 172.21.15.49:0/61793913 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24136 (secure 0 0 0) 0x7f41f009c250 con 0x7f41f81034a0 2024-03-26T17:59:54.134 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:54.134 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:51.836345+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:59:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 -- 172.21.15.49:0/61793913 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f41e00a2f20 msgr2=0x7f41e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f41e00a2f20 0x7f41e00a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f41f8199110 tx=0x7f41e800b410 comp rx=0 tx=0).stop 2024-03-26T17:59:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 -- 172.21.15.49:0/61793913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41f81034a0 msgr2=0x7f41f819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41f81034a0 0x7f41f819f1b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009fd0 tx=0x7f41f000ef10 comp rx=0 tx=0).stop 2024-03-26T17:59:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 -- 172.21.15.49:0/61793913 shutdown_connections 2024-03-26T17:59:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f41e00a2f20 0x7f41e00a53e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41f81034a0 0x7f41f819f1b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41f8102ae0 0x7f41f8198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 --2- 172.21.15.49:0/61793913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f41f81018f0 0x7f41f8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 -- 172.21.15.49:0/61793913 >> 172.21.15.49:0/61793913 conn(0x7f41f80fcea0 msgr2=0x7f41f8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:54.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 -- 172.21.15.49:0/61793913 shutdown_connections 2024-03-26T17:59:54.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.153+0000 7f41fdcf1640 1 -- 172.21.15.49:0/61793913 wait complete. 2024-03-26T17:59:54.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-26T17:59:46.796381+0000', 'modified': '2024-03-26T17:59:51.836514+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24533}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6839/1435040687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 1435040687}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 1435040687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24533, 'qdb_cluster': [24533]} 2024-03-26T17:59:54.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-26T17:59:54.149 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T17:59:54.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.275+0000 7f315892f640 1 -- 172.21.15.49:0/1095118763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3154102360 msgr2=0x7f3154104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.275+0000 7f315892f640 1 --2- 172.21.15.49:0/1095118763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3154102360 0x7f3154104750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3144009900 tx=0x7f314402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:54.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.275+0000 7f315892f640 1 -- 172.21.15.49:0/1095118763 shutdown_connections 2024-03-26T17:59:54.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.275+0000 7f315892f640 1 --2- 172.21.15.49:0/1095118763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31541075c0 0x7f3154109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.275+0000 7f315892f640 1 --2- 172.21.15.49:0/1095118763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3154104c90 0x7f3154107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.275+0000 7f315892f640 1 --2- 172.21.15.49:0/1095118763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3154102360 0x7f3154104750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.275+0000 7f315892f640 1 -- 172.21.15.49:0/1095118763 >> 172.21.15.49:0/1095118763 conn(0x7f31540fbd70 msgr2=0x7f31540fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:54.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.275+0000 7f315892f640 1 -- 172.21.15.49:0/1095118763 shutdown_connections 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.275+0000 7f315892f640 1 -- 172.21.15.49:0/1095118763 wait complete. 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f315892f640 1 Processor -- start 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f315892f640 1 -- start start 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f315892f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3154102360 0x7f3154197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f315892f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3154104c90 0x7f31541981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f315892f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31541075c0 0x7f315419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f315892f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f315410d5d0 con 0x7f3154104c90 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f315892f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f315410d450 con 0x7f31541075c0 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f315892f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f315410d750 con 0x7f3154102360 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f3151d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3154104c90 0x7f31541981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f3151d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3154104c90 0x7f31541981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59116/0 (socket says 172.21.15.49:59116) 2024-03-26T17:59:54.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f3151d9b640 1 -- 172.21.15.49:0/2533959246 learned_addr learned my addr 172.21.15.49:0/2533959246 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f3152d9d640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31541075c0 0x7f315419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.276+0000 7f315259c640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3154102360 0x7f3154197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f3151d9b640 1 -- 172.21.15.49:0/2533959246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3154102360 msgr2=0x7f3154197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f3151d9b640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3154102360 0x7f3154197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f3151d9b640 1 -- 172.21.15.49:0/2533959246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31541075c0 msgr2=0x7f315419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f3151d9b640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31541075c0 0x7f315419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f3151d9b640 1 -- 172.21.15.49:0/2533959246 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315419f8c0 con 0x7f3154104c90 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f3151d9b640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3154104c90 0x7f31541981c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f314800e9c0 tx=0x7f314800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f31437fe640 1 -- 172.21.15.49:0/2533959246 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314800cd80 con 0x7f3154104c90 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f315892f640 1 -- 172.21.15.49:0/2533959246 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315419fbb0 con 0x7f3154104c90 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f31437fe640 1 -- 172.21.15.49:0/2533959246 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3148005500 con 0x7f3154104c90 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f31437fe640 1 -- 172.21.15.49:0/2533959246 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148010690 con 0x7f3154104c90 2024-03-26T17:59:54.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.277+0000 7f315892f640 1 -- 172.21.15.49:0/2533959246 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31541a0150 con 0x7f3154104c90 2024-03-26T17:59:54.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.279+0000 7f31437fe640 1 -- 172.21.15.49:0/2533959246 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f31480046c0 con 0x7f3154104c90 2024-03-26T17:59:54.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.279+0000 7f315b5f1740 1 Processor -- start 2024-03-26T17:59:54.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315b5f1740 1 -- start start 2024-03-26T17:59:54.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f2c40 0x559fc60f3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f3e10 0x559fc60f4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 0x559fc60fb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315b5f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559fc6057270 con 0x559fc60f47d0 2024-03-26T17:59:54.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315b5f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559fc5fffc80 con 0x559fc60f2c40 2024-03-26T17:59:54.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315b5f1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559fc6053220 con 0x559fc60f3e10 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f31437fe640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f313c0a2f20 0x7f313c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f3151d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 0x559fc60fb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f3152d9d640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f313c0a2f20 0x7f313c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f3e10 0x559fc60f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f3e10 0x559fc60f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:57540/0 (socket says 172.21.15.49:57540) 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315259c640 1 -- 172.21.15.49:0/509823121 learned_addr learned my addr 172.21.15.49:0/509823121 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f3152d9d640 1 --2- 172.21.15.49:0/509823121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f2c40 0x559fc60f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f31437fe640 1 -- 172.21.15.49:0/2533959246 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f31480cf930 con 0x7f3154104c90 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315259c640 1 -- 172.21.15.49:0/509823121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f2c40 msgr2=0x559fc60f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315259c640 1 --2- 172.21.15.49:0/509823121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f2c40 0x559fc60f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315259c640 1 -- 172.21.15.49:0/509823121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 msgr2=0x559fc60fb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315259c640 1 --2- 172.21.15.49:0/509823121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 0x559fc60fb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.280+0000 7f315259c640 1 -- 172.21.15.49:0/509823121 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc6011fb0 con 0x559fc60f3e10 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.281+0000 7f3152d9d640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f313c0a2f20 0x7f313c0a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f314c009fd0 tx=0x7f314c009ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.281+0000 7f315259c640 1 --2- 172.21.15.49:0/509823121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f3e10 0x559fc60f4290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f314402c7b0 tx=0x7f3144005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f189c77d52ecacac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.281+0000 7f3140ff9640 1 -- 172.21.15.49:0/509823121 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144042070 con 0x559fc60f3e10 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.281+0000 7f3140ff9640 1 -- 172.21.15.49:0/509823121 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314402ccc0 con 0x559fc60f3e10 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.281+0000 7f3140ff9640 1 -- 172.21.15.49:0/509823121 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144004920 con 0x559fc60f3e10 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.281+0000 7f315b5f1740 1 -- 172.21.15.49:0/509823121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f3e10 msgr2=0x559fc60f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.281+0000 7f315b5f1740 1 --2- 172.21.15.49:0/509823121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f3e10 0x559fc60f4290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f314402c7b0 tx=0x7f3144005d50 comp rx=0 tx=0).stop 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 -- 172.21.15.49:0/509823121 shutdown_connections 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 --2- 172.21.15.49:0/509823121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f3e10 0x559fc60f4290 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 --2- 172.21.15.49:0/509823121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 0x559fc60fb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 --2- 172.21.15.49:0/509823121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f2c40 0x559fc60f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 -- 172.21.15.49:0/509823121 >> 172.21.15.49:0/509823121 conn(0x559fc60ee030 msgr2=0x559fc60ee430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 -- 172.21.15.49:0/509823121 shutdown_connections 2024-03-26T17:59:54.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 -- 172.21.15.49:0/509823121 wait complete. 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 -- start start 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 Processor -- start 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f2c40 0x559fc60fd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 0x559fc60fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f47d0 0x559fc61046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559fc60fb780 con 0x559fc60f2c40 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559fc6012fc0 con 0x559fc60f3e10 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559fc60fb900 con 0x559fc60f47d0 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315b5f1740 10 client.0 _ll_register_callbacks cb 0x559fc600d770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f2c40 0x559fc60fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f3151d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 0x559fc60fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f2c40 0x559fc60fd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59120/0 (socket says 172.21.15.49:59120) 2024-03-26T17:59:54.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f315259c640 1 -- 172.21.15.49:0/2205912181 learned_addr learned my addr 172.21.15.49:0/2205912181 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.282+0000 7f3152d9d640 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f47d0 0x559fc61046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3151d9b640 1 -- 172.21.15.49:0/2205912181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f47d0 msgr2=0x559fc61046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3151d9b640 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f47d0 0x559fc61046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3151d9b640 1 -- 172.21.15.49:0/2205912181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f2c40 msgr2=0x559fc60fd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3151d9b640 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f2c40 0x559fc60fd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3151d9b640 1 -- 172.21.15.49:0/2205912181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc5d6b2d0 con 0x559fc60f3e10 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f315259c640 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f2c40 0x559fc60fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3151d9b640 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 0x559fc60fd690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f31480987f0 tx=0x7f314806aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e37b0539b550977 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3139ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3139ffb640 1 -- 172.21.15.49:0/2205912181 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31480abfc0 con 0x559fc60f3e10 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f315b5f1740 1 -- 172.21.15.49:0/2205912181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f314402ccc0 con 0x559fc60f3e10 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3139ffb640 1 -- 172.21.15.49:0/2205912181 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f31480a43a0 con 0x559fc60f3e10 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3139ffb640 1 -- 172.21.15.49:0/2205912181 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f314809c520 con 0x559fc60f3e10 2024-03-26T17:59:54.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f315b5f1740 10 client.24641 fetch_fsmap learned FSMap version 17 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f315b5f1740 1 -- 172.21.15.49:0/2205912181 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x559fc5e356b0 con 0x559fc60f3e10 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3139ffb640 1 -- 172.21.15.49:0/2205912181 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314402ccc0 con 0x559fc60f3e10 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f3139ffb640 1 -- 172.21.15.49:0/2205912181 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f314806ebc0 con 0x559fc60f3e10 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f315b5f1740 10 client.24641 fetch_fsmap finished waiting for FSMap version 17 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.283+0000 7f315b5f1740 10 client.24641 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.284+0000 7f315b5f1740 20 client.24641 populate_metadata read hostname 'smithi049' 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.284+0000 7f315b5f1740 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x559fc6109ec0 0x559fc610b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.284+0000 7f315b5f1740 4 client.24641 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.284+0000 7f315b5f1740 1 -- 172.21.15.49:0/2205912181 --> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fc60647b0 con 0x559fc6109ec0 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.284+0000 7f315259c640 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x559fc6109ec0 0x559fc610b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.284+0000 7f3139ffb640 10 client.24641 ms_handle_connect on v2:172.21.15.115:6838/1435040687 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.284+0000 7f315259c640 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x559fc6109ec0 0x559fc610b330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7e7f4501b3e882 server_cookie=ec1fe0e3914f309a in_seq=0 out_seq=0 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f3139ffb640 1 -- 172.21.15.49:0/2205912181 <== mds.0 v2:172.21.15.115:6838/1435040687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559fc60647b0 con 0x559fc6109ec0 2024-03-26T17:59:54.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f3139ffb640 10 client.24641 handle_command_reply: tid=0 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 client.24641 shutdown 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 -- 172.21.15.49:0/2205912181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 msgr2=0x559fc60fd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 0x559fc60fd690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f31480987f0 tx=0x7f314806aa10 comp rx=0 tx=0).stop 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 -- 172.21.15.49:0/2205912181 shutdown_connections 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x559fc6109ec0 0x559fc610b330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f47d0 0x559fc61046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f3139ffb640 0 client.24641 ms_handle_reset on v2:172.21.15.115:6838/1435040687 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 0x559fc60fd690 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 --2- 172.21.15.49:0/2205912181 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f2c40 0x559fc60fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 -- 172.21.15.49:0/2205912181 >> 172.21.15.49:0/2205912181 conn(0x559fc60ee030 msgr2=0x559fc60ed230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 -- 172.21.15.49:0/2205912181 shutdown_connections 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 1 -- 172.21.15.49:0/2205912181 wait complete. 2024-03-26T17:59:54.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.285+0000 7f315b5f1740 20 client.24641 trim_cache size 0 max 16384 2024-03-26T17:59:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f315b5f1740 1 Processor -- start 2024-03-26T17:59:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f315b5f1740 1 -- start start 2024-03-26T17:59:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f2c40 0x559fc60f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f3e10 0x559fc6104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f47d0 0x559fc60fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f315b5f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559fc6109670 con 0x559fc60f3e10 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f315b5f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559fc6109d40 con 0x559fc60f47d0 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f315b5f1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559fc610a410 con 0x559fc60f2c40 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f3151d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f3e10 0x559fc6104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f3151d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f3e10 0x559fc6104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59126/0 (socket says 172.21.15.49:59126) 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f3151d9b640 1 -- 172.21.15.49:0/3607727315 learned_addr learned my addr 172.21.15.49:0/3607727315 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f3152d9d640 1 --2- 172.21.15.49:0/3607727315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f47d0 0x559fc60fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f315259c640 1 --2- 172.21.15.49:0/3607727315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f2c40 0x559fc60f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f3151d9b640 1 -- 172.21.15.49:0/3607727315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f2c40 msgr2=0x559fc60f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.303+0000 7f3151d9b640 1 --2- 172.21.15.49:0/3607727315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f2c40 0x559fc60f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f3151d9b640 1 -- 172.21.15.49:0/3607727315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f47d0 msgr2=0x559fc60fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f3151d9b640 1 --2- 172.21.15.49:0/3607727315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f47d0 0x559fc60fcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f3151d9b640 1 -- 172.21.15.49:0/3607727315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc60f3890 con 0x559fc60f3e10 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f3151d9b640 1 --2- 172.21.15.49:0/3607727315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f3e10 0x559fc6104690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3148000c00 tx=0x7f3148009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cbf0a55f1bab1b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:54.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f313bfff640 1 -- 172.21.15.49:0/3607727315 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314809b960 con 0x559fc60f3e10 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f313bfff640 1 -- 172.21.15.49:0/3607727315 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314809be20 con 0x559fc60f3e10 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f313bfff640 1 -- 172.21.15.49:0/3607727315 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314809b270 con 0x559fc60f3e10 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f315b5f1740 1 -- 172.21.15.49:0/3607727315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f3e10 msgr2=0x559fc6104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f315b5f1740 1 --2- 172.21.15.49:0/3607727315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f3e10 0x559fc6104690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3148000c00 tx=0x7f3148009700 comp rx=0 tx=0).stop 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f315b5f1740 1 -- 172.21.15.49:0/3607727315 shutdown_connections 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f315b5f1740 1 --2- 172.21.15.49:0/3607727315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f47d0 0x559fc60fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f315b5f1740 1 --2- 172.21.15.49:0/3607727315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f3e10 0x559fc6104690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f315b5f1740 1 --2- 172.21.15.49:0/3607727315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f2c40 0x559fc60f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f315b5f1740 1 -- 172.21.15.49:0/3607727315 >> 172.21.15.49:0/3607727315 conn(0x559fc60ee030 msgr2=0x559fc60ee410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f315b5f1740 1 -- 172.21.15.49:0/3607727315 shutdown_connections 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.304+0000 7f315b5f1740 1 -- 172.21.15.49:0/3607727315 wait complete. 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f315b5f1740 1 -- start start 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f315b5f1740 1 Processor -- start 2024-03-26T17:59:54.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f2c40 0x559fc61006c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 0x559fc6100c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f315b5f1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 0x559fc6103b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f315b5f1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x559fc610b1b0 con 0x559fc60f47d0 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f315b5f1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x559fc610aae0 con 0x559fc60f3e10 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f315b5f1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559fc610b880 con 0x559fc60f2c40 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f315b5f1740 10 client.0 _ll_register_callbacks cb 0x7f31480056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f3152d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 0x559fc6103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f3151d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 0x559fc6100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f3152d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 0x559fc6103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59130/0 (socket says 172.21.15.49:59130) 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f3152d9d640 1 -- 172.21.15.49:0/424245187 learned_addr learned my addr 172.21.15.49:0/424245187 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f315259c640 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f2c40 0x559fc61006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f3151d9b640 1 -- 172.21.15.49:0/424245187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f2c40 msgr2=0x559fc61006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f3151d9b640 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f2c40 0x559fc61006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f3151d9b640 1 -- 172.21.15.49:0/424245187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 msgr2=0x559fc6103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f3151d9b640 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 0x559fc6103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f3151d9b640 1 -- 172.21.15.49:0/424245187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc6104100 con 0x559fc60f3e10 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.305+0000 7f3152d9d640 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 0x559fc6103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f3151d9b640 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 0x559fc6100c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x559fc60ee000 tx=0x7f31480057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc912d0a0c3e7d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f3139ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f3139ffb640 1 -- 172.21.15.49:0/424245187 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314809b960 con 0x559fc60f3e10 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f315b5f1740 1 -- 172.21.15.49:0/424245187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f314809be20 con 0x559fc60f3e10 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f3139ffb640 1 -- 172.21.15.49:0/424245187 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f314806dd10 con 0x559fc60f3e10 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f3139ffb640 1 -- 172.21.15.49:0/424245187 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f314809ff70 con 0x559fc60f3e10 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f315b5f1740 10 client.24647 fetch_fsmap learned FSMap version 17 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f315b5f1740 1 -- 172.21.15.49:0/424245187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x559fc6170470 con 0x559fc60f3e10 2024-03-26T17:59:54.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f3139ffb640 1 -- 172.21.15.49:0/424245187 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f314809be20 con 0x559fc60f3e10 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f3139ffb640 1 -- 172.21.15.49:0/424245187 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f3148062030 con 0x559fc60f3e10 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f315b5f1740 10 client.24647 fetch_fsmap finished waiting for FSMap version 17 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f315b5f1740 10 client.24647 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f315b5f1740 20 client.24647 populate_metadata read hostname 'smithi049' 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f315b5f1740 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x559fc6174d60 0x559fc6171870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f315b5f1740 4 client.24647 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f315b5f1740 1 -- 172.21.15.49:0/424245187 --> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] -- command(tid 0: {"prefix": "status"}) -- 0x559fc605eb00 con 0x559fc6174d60 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.306+0000 7f315259c640 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x559fc6174d60 0x559fc6171870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315259c640 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x559fc6174d60 0x559fc6171870 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a7db8bb1de0cc88 server_cookie=4f4fc1e13d4585fd in_seq=0 out_seq=0 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f3139ffb640 10 client.24647 ms_handle_connect on v2:172.21.15.115:6838/1435040687 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f3139ffb640 1 -- 172.21.15.49:0/424245187 <== mds.0 v2:172.21.15.115:6838/1435040687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559fc605eb00 con 0x559fc6174d60 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f3139ffb640 10 client.24647 handle_command_reply: tid=0 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 client.24647 shutdown 2024-03-26T17:59:54.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 -- 172.21.15.49:0/424245187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 msgr2=0x559fc6100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 0x559fc6100c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x559fc60ee000 tx=0x7f31480057b0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 -- 172.21.15.49:0/424245187 shutdown_connections 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x559fc6174d60 0x559fc6171870 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fc60f47d0 0x559fc6103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f3139ffb640 0 client.24647 ms_handle_reset on v2:172.21.15.115:6838/1435040687 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559fc60f3e10 0x559fc6100c00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 --2- 172.21.15.49:0/424245187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc60f2c40 0x559fc61006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 -- 172.21.15.49:0/424245187 >> 172.21.15.49:0/424245187 conn(0x559fc60ee030 msgr2=0x559fc60fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 -- 172.21.15.49:0/424245187 shutdown_connections 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 1 -- 172.21.15.49:0/424245187 wait complete. 2024-03-26T17:59:54.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.307+0000 7f315b5f1740 20 client.24647 trim_cache size 0 max 16384 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24533, 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 6.8791402399999999, 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 17, 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 90, 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T17:59:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 11.006173971000001 2024-03-26T17:59:54.299 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T17:59:54.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 -- 172.21.15.49:0/2533959246 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f313c0a2f20 msgr2=0x7f313c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f313c0a2f20 0x7f313c0a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f314c009fd0 tx=0x7f314c009ee0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 -- 172.21.15.49:0/2533959246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3154104c90 msgr2=0x7f31541981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:54.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3154104c90 0x7f31541981c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f314800e9c0 tx=0x7f314800ee80 comp rx=0 tx=0).stop 2024-03-26T17:59:54.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 -- 172.21.15.49:0/2533959246 shutdown_connections 2024-03-26T17:59:54.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f313c0a2f20 0x7f313c0a53e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31541075c0 0x7f315419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3154104c90 0x7f31541981c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 --2- 172.21.15.49:0/2533959246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3154102360 0x7f3154197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:54.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 -- 172.21.15.49:0/2533959246 >> 172.21.15.49:0/2533959246 conn(0x7f31540fbd70 msgr2=0x7f3154102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:54.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 -- 172.21.15.49:0/2533959246 shutdown_connections 2024-03-26T17:59:54.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:54.308+0000 7f315892f640 1 -- 172.21.15.49:0/2533959246 wait complete. 2024-03-26T17:59:54.301 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-26T17:59:54.301 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-26T17:59:54.301 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-26T17:59:54.549 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:59:54.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:54 smithi115 ceph-mon[36174]: pgmap v295: 129 pgs: 4 creating+peering, 113 active+clean, 12 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-26T17:59:54.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:54 smithi115 ceph-mon[36174]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T17:59:54.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:54 smithi115 ceph-mon[36174]: Cluster is now healthy 2024-03-26T17:59:54.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:54 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3592696576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:54.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:54 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/12657815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:54.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:54 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/61793913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:54.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:54 smithi049 ceph-mon[32331]: pgmap v295: 129 pgs: 4 creating+peering, 113 active+clean, 12 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-26T17:59:54.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:54 smithi049 ceph-mon[32331]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T17:59:54.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:54 smithi049 ceph-mon[32331]: Cluster is now healthy 2024-03-26T17:59:54.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:54 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3592696576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:54.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:54 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/12657815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:54.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:54 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/61793913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:54.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:54 smithi171 ceph-mon[36033]: pgmap v295: 129 pgs: 4 creating+peering, 113 active+clean, 12 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-26T17:59:54.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:54 smithi171 ceph-mon[36033]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T17:59:54.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:54 smithi171 ceph-mon[36033]: Cluster is now healthy 2024-03-26T17:59:54.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:54 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3592696576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:54.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:54 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/12657815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:54.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:54 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/61793913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:59:56.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.203+0000 7fc1aead7700 1 -- 172.21.15.49:0/1876349498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1a810ee90 msgr2=0x7fc1a8111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.203+0000 7fc1aead7700 1 --2- 172.21.15.49:0/1876349498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1a810ee90 0x7fc1a8111280 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000b0a0 tx=0x7fc1a000b3b0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.203+0000 7fc1aead7700 1 -- 172.21.15.49:0/1876349498 shutdown_connections 2024-03-26T17:59:56.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.203+0000 7fc1aead7700 1 --2- 172.21.15.49:0/1876349498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1a810ee90 0x7fc1a8111280 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.203+0000 7fc1aead7700 1 --2- 172.21.15.49:0/1876349498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a806aab0 0x7fc1a806ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.203+0000 7fc1aead7700 1 --2- 172.21.15.49:0/1876349498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1a806b3d0 0x7fc1a810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.203+0000 7fc1aead7700 1 -- 172.21.15.49:0/1876349498 >> 172.21.15.49:0/1876349498 conn(0x7fc1a80772d0 msgr2=0x7fc1a80776e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.203+0000 7fc1aead7700 1 -- 172.21.15.49:0/1876349498 shutdown_connections 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.203+0000 7fc1aead7700 1 -- 172.21.15.49:0/1876349498 wait complete. 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1aead7700 1 Processor -- start 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1aead7700 1 -- start start 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1aead7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1a806aab0 0x7fc1a819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1aead7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1a806b3d0 0x7fc1a819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1aead7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a810ee90 0x7fc1a819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1aead7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8114950 con 0x7fc1a806b3d0 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1aead7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81147d0 con 0x7fc1a810ee90 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1aead7700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8114ad0 con 0x7fc1a806aab0 2024-03-26T17:59:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1a7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1a806b3d0 0x7fc1a819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1a7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1a806b3d0 0x7fc1a819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59146/0 (socket says 172.21.15.49:59146) 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1a7fff700 1 -- 172.21.15.49:0/1546108534 learned_addr learned my addr 172.21.15.49:0/1546108534 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1ad074700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a810ee90 0x7fc1a819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.204+0000 7fc1ac873700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1a806aab0 0x7fc1a819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1ad074700 1 -- 172.21.15.49:0/1546108534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1a806aab0 msgr2=0x7fc1a819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1ad074700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1a806aab0 0x7fc1a819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1ad074700 1 -- 172.21.15.49:0/1546108534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1a806b3d0 msgr2=0x7fc1a819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1ad074700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1a806b3d0 0x7fc1a819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1ad074700 1 -- 172.21.15.49:0/1546108534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a819fde0 con 0x7fc1a810ee90 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1a7fff700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1a806b3d0 0x7fc1a819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1ac873700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1a806aab0 0x7fc1a819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1ad074700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a810ee90 0x7fc1a819f6a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000b070 tx=0x7fc1a0004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:56.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1a5ffb700 1 -- 172.21.15.49:0/1546108534 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0021070 con 0x7fc1a810ee90 2024-03-26T17:59:56.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1a5ffb700 1 -- 172.21.15.49:0/1546108534 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a0007b80 con 0x7fc1a810ee90 2024-03-26T17:59:56.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1aead7700 1 -- 172.21.15.49:0/1546108534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a81a0070 con 0x7fc1a810ee90 2024-03-26T17:59:56.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1a5ffb700 1 -- 172.21.15.49:0/1546108534 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00052c0 con 0x7fc1a810ee90 2024-03-26T17:59:56.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.205+0000 7fc1aead7700 1 -- 172.21.15.49:0/1546108534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a81a7950 con 0x7fc1a810ee90 2024-03-26T17:59:56.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.206+0000 7fc1aead7700 1 -- 172.21.15.49:0/1546108534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a81018c0 con 0x7fc1a810ee90 2024-03-26T17:59:56.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.207+0000 7fc1a5ffb700 1 -- 172.21.15.49:0/1546108534 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fc1a0024070 con 0x7fc1a810ee90 2024-03-26T17:59:56.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.208+0000 7fc1a5ffb700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc1940a2fb0 0x7fc1940a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.208+0000 7fc1ac873700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc1940a2fb0 0x7fc1940a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.208+0000 7fc1a5ffb700 1 -- 172.21.15.49:0/1546108534 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc1a000f020 con 0x7fc1a810ee90 2024-03-26T17:59:56.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.208+0000 7fc1ac873700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc1940a2fb0 0x7fc1940a5470 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc190009cb0 tx=0x7fc190009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:56.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.212+0000 7fc1a5ffb700 1 -- 172.21.15.49:0/1546108534 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc1a002b020 con 0x7fc1a810ee90 2024-03-26T17:59:56.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.465+0000 7fc1aead7700 1 -- 172.21.15.49:0/1546108534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc1a802dc00 con 0x7fc1a810ee90 2024-03-26T17:59:56.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.467+0000 7fc1a5ffb700 1 -- 172.21.15.49:0/1546108534 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fc1a009dc20 con 0x7fc1a810ee90 2024-03-26T17:59:56.449 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-26T17:59:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 -- 172.21.15.49:0/1546108534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc1940a2fb0 msgr2=0x7fc1940a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc1940a2fb0 0x7fc1940a5470 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc190009cb0 tx=0x7fc190009260 comp rx=0 tx=0).stop 2024-03-26T17:59:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 -- 172.21.15.49:0/1546108534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a810ee90 msgr2=0x7fc1a819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a810ee90 0x7fc1a819f6a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000b070 tx=0x7fc1a0004c80 comp rx=0 tx=0).stop 2024-03-26T17:59:56.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 -- 172.21.15.49:0/1546108534 shutdown_connections 2024-03-26T17:59:56.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc1940a2fb0 0x7fc1940a5470 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1a810ee90 0x7fc1a819f6a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1a806aab0 0x7fc1a819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 --2- 172.21.15.49:0/1546108534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1a806b3d0 0x7fc1a819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 -- 172.21.15.49:0/1546108534 >> 172.21.15.49:0/1546108534 conn(0x7fc1a80772d0 msgr2=0x7fc1a81102e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:56.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 -- 172.21.15.49:0/1546108534 shutdown_connections 2024-03-26T17:59:56.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:56.469+0000 7fc1aead7700 1 -- 172.21.15.49:0/1546108534 wait complete. 2024-03-26T17:59:56.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:56 smithi115 ceph-mon[36174]: pgmap v296: 129 pgs: 4 creating+peering, 117 active+clean, 8 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-03-26T17:59:56.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:56 smithi049 ceph-mon[32331]: pgmap v296: 129 pgs: 4 creating+peering, 117 active+clean, 8 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-03-26T17:59:56.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:56 smithi171 ceph-mon[36033]: pgmap v296: 129 pgs: 4 creating+peering, 117 active+clean, 8 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-03-26T17:59:57.114 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-26T17:59:57.114 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-26T17:59:57.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-03-26T17:59:57.115 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-26T17:59:57.115 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-26T17:59:57.115 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-26T17:59:57.115 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-26T17:59:57.115 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-26T17:59:57.115 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-26T17:59:57.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:57.115 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-03-26T17:59:57.133 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-26T17:59:57.133 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-26T17:59:57.133 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-03-26T17:59:57.134 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-26T17:59:57.135 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42207sec preferred_lft 42207sec 2024-03-26T17:59:57.135 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-03-26T17:59:57.135 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-26T17:59:57.135 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T17:59:57.135 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-03-26T17:59:57.135 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-03-26T17:59:57.136 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-26T17:59:57.136 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T17:59:57.136 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-26T17:59:57.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2024-03-26T17:59:57.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2024-03-26T17:59:57.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2024-03-26T17:59:57.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-26T17:59:57.136 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-26T17:59:57.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T17:59:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:59:57.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:57.357 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-26T17:59:57.387 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-03-26T17:59:57.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:57.389 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-03-26T17:59:57.460 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-03-26T17:59:57.460 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-26T17:59:57.460 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-26T17:59:57.460 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-26T17:59:57.461 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-26T17:59:57.461 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T17:59:57.461 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-26T17:59:57.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-26T17:59:57.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-26T17:59:57.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-26T17:59:57.461 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-26T17:59:57.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T17:59:57.642 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:57 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1546108534' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T17:59:57.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:57 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1546108534' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T17:59:57.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:59:57.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:57.752 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-03-26T17:59:57.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:57.768 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-03-26T17:59:57.823 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T17:59:57.823 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-26T17:59:57.823 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T17:59:57.824 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-26T17:59:57.824 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-26T17:59:57.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:57 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1546108534' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T17:59:57.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T17:59:57.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:59:57.955 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-26T17:59:57.955 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T17:59:57.955 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-26T17:59:57.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-26T17:59:57.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-26T17:59:57.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-26T17:59:57.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-26T17:59:57.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-26T17:59:57.955 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-26T17:59:58.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:59:58.113 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T17:59:58.113 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-26T17:59:58.113 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-03-26T17:59:58.113 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2024-03-26T17:59:58.113 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-26T17:59:58.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T17:59:58.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:59:58.206 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-26T17:59:58.206 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T17:59:58.206 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-26T17:59:58.221 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-26T17:59:58.221 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T17:59:58.221 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-26T17:59:58.272 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2024-03-26T17:59:58.343 DEBUG:teuthology.orchestra.run.smithi049:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T17:59:58.401 INFO:teuthology.orchestra.run.smithi049.stdout:/proc 2024-03-26T17:59:58.401 INFO:teuthology.orchestra.run.smithi049.stdout:/sys 2024-03-26T17:59:58.402 INFO:teuthology.orchestra.run.smithi049.stdout:/dev 2024-03-26T17:59:58.402 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/security 2024-03-26T17:59:58.402 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/shm 2024-03-26T17:59:58.402 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/pts 2024-03-26T17:59:58.402 INFO:teuthology.orchestra.run.smithi049.stdout:/run 2024-03-26T17:59:58.402 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup 2024-03-26T17:59:58.402 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/pstore 2024-03-26T17:59:58.403 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/bpf 2024-03-26T17:59:58.403 INFO:teuthology.orchestra.run.smithi049.stdout:/ 2024-03-26T17:59:58.403 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/selinux 2024-03-26T17:59:58.403 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T17:59:58.403 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/mqueue 2024-03-26T17:59:58.403 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/hugepages 2024-03-26T17:59:58.404 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/debug 2024-03-26T17:59:58.404 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/tracing 2024-03-26T17:59:58.404 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/fuse/connections 2024-03-26T17:59:58.404 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/config 2024-03-26T17:59:58.404 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T17:59:58.404 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T17:59:58.404 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T17:59:58.404 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T17:59:58.404 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/1000 2024-03-26T17:59:58.405 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/ceph 2024-03-26T17:59:58.405 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay 2024-03-26T17:59:58.405 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/33e4b53398b1d5728aa4d5db8cec76bae33af2069be3310328d72487d97e1531/merged 2024-03-26T17:59:58.405 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/441ace45b83ed714b99c4da42b79b74d09a9ba6733d0f2051158825ea3d62490/merged 2024-03-26T17:59:58.405 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/0 2024-03-26T17:59:58.405 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T17:59:58.405 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/d215da0fecb05c1d721f2590e08077174450af3ff9fcbadc5864be69a73be565/merged 2024-03-26T17:59:58.406 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/a3252d087fd498853d13003124cbf319a0e45e5922d9c2106914ba24c6fd6e4a/merged 2024-03-26T17:59:58.406 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/de23af85cf29bfa53d81eef6912cbea193e5432762a262dc1af08b1582805649/merged 2024-03-26T17:59:58.406 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/6d4ab6b10596975cd2d5401262dfc226069d624ebf17620220a8059db270e888/merged 2024-03-26T17:59:58.406 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/db763361e33c2bb693bf789b31cafbcfb5dbcbc04ba5d0f2d665061c4eb1c9da/merged 2024-03-26T17:59:58.406 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/9917b85d8b35bb90523b32f3ab88ff0c2e319e75ad4996fc2f80c8a4a277543b/merged 2024-03-26T17:59:58.406 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/555e17beca942acfeac848536b9a5abb77e3fdcec104fce83fcf598001a0b676/merged 2024-03-26T17:59:58.406 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/b5cfcf64f161444b762e9bb6a708f97b20375a9ad654db70b6e815b799ffce54/merged 2024-03-26T17:59:58.407 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns 2024-03-26T17:59:58.407 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T17:59:58.409 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T17:59:58.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:58.410 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2024-03-26T17:59:58.457 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-26T17:59:58.458 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-26T17:59:58.501 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2024-03-26T17:59:58.532 DEBUG:teuthology.orchestra.run.smithi049:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T17:59:58.589 INFO:teuthology.orchestra.run.smithi049.stdout:/proc 2024-03-26T17:59:58.589 INFO:teuthology.orchestra.run.smithi049.stdout:/sys 2024-03-26T17:59:58.589 INFO:teuthology.orchestra.run.smithi049.stdout:/dev 2024-03-26T17:59:58.589 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/security 2024-03-26T17:59:58.589 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/shm 2024-03-26T17:59:58.590 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/pts 2024-03-26T17:59:58.590 INFO:teuthology.orchestra.run.smithi049.stdout:/run 2024-03-26T17:59:58.590 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup 2024-03-26T17:59:58.590 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/pstore 2024-03-26T17:59:58.590 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/bpf 2024-03-26T17:59:58.590 INFO:teuthology.orchestra.run.smithi049.stdout:/ 2024-03-26T17:59:58.590 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/selinux 2024-03-26T17:59:58.591 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T17:59:58.591 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/mqueue 2024-03-26T17:59:58.591 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/hugepages 2024-03-26T17:59:58.591 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/debug 2024-03-26T17:59:58.591 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/tracing 2024-03-26T17:59:58.591 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/fuse/connections 2024-03-26T17:59:58.591 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/config 2024-03-26T17:59:58.591 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T17:59:58.592 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T17:59:58.592 INFO:teuthology.orchestra.run.smithi049.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T17:59:58.592 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T17:59:58.592 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/1000 2024-03-26T17:59:58.592 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/ceph 2024-03-26T17:59:58.592 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay 2024-03-26T17:59:58.592 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/33e4b53398b1d5728aa4d5db8cec76bae33af2069be3310328d72487d97e1531/merged 2024-03-26T17:59:58.593 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/441ace45b83ed714b99c4da42b79b74d09a9ba6733d0f2051158825ea3d62490/merged 2024-03-26T17:59:58.593 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/0 2024-03-26T17:59:58.593 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T17:59:58.593 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/d215da0fecb05c1d721f2590e08077174450af3ff9fcbadc5864be69a73be565/merged 2024-03-26T17:59:58.593 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/a3252d087fd498853d13003124cbf319a0e45e5922d9c2106914ba24c6fd6e4a/merged 2024-03-26T17:59:58.593 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/de23af85cf29bfa53d81eef6912cbea193e5432762a262dc1af08b1582805649/merged 2024-03-26T17:59:58.593 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/6d4ab6b10596975cd2d5401262dfc226069d624ebf17620220a8059db270e888/merged 2024-03-26T17:59:58.593 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/db763361e33c2bb693bf789b31cafbcfb5dbcbc04ba5d0f2d665061c4eb1c9da/merged 2024-03-26T17:59:58.594 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/9917b85d8b35bb90523b32f3ab88ff0c2e319e75ad4996fc2f80c8a4a277543b/merged 2024-03-26T17:59:58.594 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/555e17beca942acfeac848536b9a5abb77e3fdcec104fce83fcf598001a0b676/merged 2024-03-26T17:59:58.594 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/b5cfcf64f161444b762e9bb6a708f97b20375a9ad654db70b6e815b799ffce54/merged 2024-03-26T17:59:58.594 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns 2024-03-26T17:59:58.594 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T17:59:58.594 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T17:59:58.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:58.595 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2024-03-26T17:59:58.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi049.stderr:2024-03-26T17:59:58.626+0000 7fbfbd160480 -1 init, newargv = 0x55e7e75b3670 newargc=15 2024-03-26T17:59:58.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi049.stderr:ceph-fuse[75942]: starting ceph client 2024-03-26T17:59:58.614 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi049.stderr:ceph-fuse[75942]: starting fuse 2024-03-26T17:59:58.625 INFO:teuthology.orchestra.run.smithi049.stdout:119 2024-03-26T17:59:58.625 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo stdin-killer -- python3 -c ' 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> import re 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> import os 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-26T17:59:58.626 DEBUG:teuthology.orchestra.run.smithi049:> 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> return f 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> 2024-03-26T17:59:58.627 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.admin")) 2024-03-26T17:59:58.628 DEBUG:teuthology.orchestra.run.smithi049:> ' 2024-03-26T17:59:58.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 17:59:58 smithi115 ceph-mon[36174]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 163 B/s wr, 3 op/s 2024-03-26T17:59:58.751 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.admin.75942.asok 2024-03-26T17:59:58.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 17:59:58 smithi049 ceph-mon[32331]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 163 B/s wr, 3 op/s 2024-03-26T17:59:58.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:59:58.761 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75942.asok 2024-03-26T17:59:58.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:58.762 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75942.asok status 2024-03-26T17:59:58.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 17:59:58 smithi171 ceph-mon[36033]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 163 B/s wr, 3 op/s 2024-03-26T17:59:58.904 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T17:59:58.904 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2024-03-26T17:59:58.904 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "360516069d9393362c4cc6eb9371680fe16d66ab", 2024-03-26T17:59:58.904 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T17:59:58.904 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "admin", 2024-03-26T17:59:58.904 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2024-03-26T17:59:58.904 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "75942", 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 14760, 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 14760 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2024-03-26T17:59:58.905 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1341091635 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1341091635 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.14760 192.168.0.1:0/1341091635", 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/1341091635", 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2024-03-26T17:59:58.906 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 17, 2024-03-26T17:59:58.907 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 90, 2024-03-26T17:59:58.907 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2024-03-26T17:59:58.907 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2024-03-26T17:59:58.907 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2024-03-26T17:59:58.907 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T17:59:58.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:58.912 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2024-03-26T17:59:58.913 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-26T17:59:58.928 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2024-03-26T17:59:58.929 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-26T17:59:58.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:58.929 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-26T17:59:59.001 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-26T17:59:59.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-26T17:59:59.106 INFO:teuthology.orchestra.run.smithi049.stdout:# file: . 2024-03-26T17:59:59.106 INFO:teuthology.orchestra.run.smithi049.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-26T17:59:59.107 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:59.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:59:59.112 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T17:59:59.112 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T17:59:59.129 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:59:59.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.295+0000 7fc543212640 1 -- 172.21.15.49:0/1700955192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc53c101a50 msgr2=0x7fc53c101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.295+0000 7fc543212640 1 --2- 172.21.15.49:0/1700955192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc53c101a50 0x7fc53c101ed0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009900 tx=0x7fc52c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:59.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.295+0000 7fc543212640 1 -- 172.21.15.49:0/1700955192 shutdown_connections 2024-03-26T17:59:59.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.295+0000 7fc543212640 1 --2- 172.21.15.49:0/1700955192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c102410 0x7fc53c108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.295+0000 7fc543212640 1 --2- 172.21.15.49:0/1700955192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc53c101a50 0x7fc53c101ed0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.295+0000 7fc543212640 1 --2- 172.21.15.49:0/1700955192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc53c100860 0x7fc53c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.295+0000 7fc543212640 1 -- 172.21.15.49:0/1700955192 >> 172.21.15.49:0/1700955192 conn(0x7fc53c0fbdf0 msgr2=0x7fc53c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.295+0000 7fc543212640 1 -- 172.21.15.49:0/1700955192 shutdown_connections 2024-03-26T17:59:59.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.295+0000 7fc543212640 1 -- 172.21.15.49:0/1700955192 wait complete. 2024-03-26T17:59:59.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc543212640 1 Processor -- start 2024-03-26T17:59:59.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc543212640 1 -- start start 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc543212640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc53c100860 0x7fc53c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc543212640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c101a50 0x7fc53c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc543212640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc53c102410 0x7fc53c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc543212640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c10c470 con 0x7fc53c100860 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc543212640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c10c2f0 con 0x7fc53c102410 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc543212640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c10c5f0 con 0x7fc53c101a50 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc540fae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc53c100860 0x7fc53c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc540fae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc53c100860 0x7fc53c197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41290/0 (socket says 172.21.15.49:41290) 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc540fae640 1 -- 172.21.15.49:0/670015112 learned_addr learned my addr 172.21.15.49:0/670015112 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc5417af640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc53c102410 0x7fc53c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc533fff640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c101a50 0x7fc53c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc540fae640 1 -- 172.21.15.49:0/670015112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c101a50 msgr2=0x7fc53c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc540fae640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c101a50 0x7fc53c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc540fae640 1 -- 172.21.15.49:0/670015112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc53c102410 msgr2=0x7fc53c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc540fae640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc53c102410 0x7fc53c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc540fae640 1 -- 172.21.15.49:0/670015112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c19f860 con 0x7fc53c100860 2024-03-26T17:59:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.296+0000 7fc5417af640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc53c102410 0x7fc53c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.297+0000 7fc540fae640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc53c100860 0x7fc53c197c50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc53800e9c0 tx=0x7fc53800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.297+0000 7fc531ffb640 1 -- 172.21.15.49:0/670015112 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53800cde0 con 0x7fc53c100860 2024-03-26T17:59:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.297+0000 7fc531ffb640 1 -- 172.21.15.49:0/670015112 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc538010910 con 0x7fc53c100860 2024-03-26T17:59:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.297+0000 7fc543212640 1 -- 172.21.15.49:0/670015112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c19fb50 con 0x7fc53c100860 2024-03-26T17:59:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.297+0000 7fc531ffb640 1 -- 172.21.15.49:0/670015112 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538018a00 con 0x7fc53c100860 2024-03-26T17:59:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.297+0000 7fc543212640 1 -- 172.21.15.49:0/670015112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c1a00a0 con 0x7fc53c100860 2024-03-26T17:59:59.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.298+0000 7fc543212640 1 -- 172.21.15.49:0/670015112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c1a0270 con 0x7fc53c100860 2024-03-26T17:59:59.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.299+0000 7fc531ffb640 1 -- 172.21.15.49:0/670015112 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fc53801f020 con 0x7fc53c100860 2024-03-26T17:59:59.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.300+0000 7fc531ffb640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc5240a2f20 0x7fc5240a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.300+0000 7fc533fff640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc5240a2f20 0x7fc5240a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.300+0000 7fc531ffb640 1 -- 172.21.15.49:0/670015112 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc5380cf890 con 0x7fc53c100860 2024-03-26T17:59:59.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.300+0000 7fc533fff640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc5240a2f20 0x7fc5240a53e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc52c02c7b0 tx=0x7fc52c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.304+0000 7fc531ffb640 1 -- 172.21.15.49:0/670015112 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc53809c480 con 0x7fc53c100860 2024-03-26T17:59:59.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.494+0000 7fc543212640 1 -- 172.21.15.49:0/670015112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc53c1a0560 con 0x7fc53c100860 2024-03-26T17:59:59.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.495+0000 7fc531ffb640 1 -- 172.21.15.49:0/670015112 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24136 (secure 0 0 0) 0x7fc53809bd80 con 0x7fc53c100860 2024-03-26T17:59:59.478 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T17:59:59.478 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:51.836345+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:59:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 -- 172.21.15.49:0/670015112 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc5240a2f20 msgr2=0x7fc5240a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc5240a2f20 0x7fc5240a53e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc52c02c7b0 tx=0x7fc52c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 -- 172.21.15.49:0/670015112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc53c100860 msgr2=0x7fc53c197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc53c100860 0x7fc53c197c50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc53800e9c0 tx=0x7fc53800ee80 comp rx=0 tx=0).stop 2024-03-26T17:59:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 -- 172.21.15.49:0/670015112 shutdown_connections 2024-03-26T17:59:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc5240a2f20 0x7fc5240a53e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c101a50 0x7fc53c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc53c102410 0x7fc53c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 --2- 172.21.15.49:0/670015112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc53c100860 0x7fc53c197c50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 -- 172.21.15.49:0/670015112 >> 172.21.15.49:0/670015112 conn(0x7fc53c0fbdf0 msgr2=0x7fc53c0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 -- 172.21.15.49:0/670015112 shutdown_connections 2024-03-26T17:59:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.497+0000 7fc543212640 1 -- 172.21.15.49:0/670015112 wait complete. 2024-03-26T17:59:59.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:59:45.382542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:59:46.221139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T17:59:49.890383+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T17:59:59.491 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:59:59.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.620+0000 7f8887753640 1 -- 172.21.15.49:0/3322097859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8880105470 msgr2=0x7f88801058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.620+0000 7f8887753640 1 --2- 172.21.15.49:0/3322097859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8880105470 0x7f88801058f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f887c009ca0 tx=0x7f887c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.620+0000 7f8887753640 1 -- 172.21.15.49:0/3322097859 shutdown_connections 2024-03-26T17:59:59.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.620+0000 7f8887753640 1 --2- 172.21.15.49:0/3322097859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8880105470 0x7f88801058f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.620+0000 7f8887753640 1 --2- 172.21.15.49:0/3322097859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8880102ac0 0x7f8880104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.620+0000 7f8887753640 1 --2- 172.21.15.49:0/3322097859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8880100150 0x7f8880102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.620+0000 7f8887753640 1 -- 172.21.15.49:0/3322097859 >> 172.21.15.49:0/3322097859 conn(0x7f88800fbd70 msgr2=0x7f88800fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.620+0000 7f8887753640 1 -- 172.21.15.49:0/3322097859 shutdown_connections 2024-03-26T17:59:59.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.620+0000 7f8887753640 1 -- 172.21.15.49:0/3322097859 wait complete. 2024-03-26T17:59:59.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f8887753640 1 Processor -- start 2024-03-26T17:59:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f8887753640 1 -- start start 2024-03-26T17:59:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f8887753640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8880100150 0x7f888019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f8887753640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8880102ac0 0x7f888019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f8887753640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8880105470 0x7f88801a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f8887753640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8880113b40 con 0x7f8880100150 2024-03-26T17:59:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f8887753640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f88801139c0 con 0x7f8880105470 2024-03-26T17:59:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f8887753640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8880113cc0 con 0x7f8880102ac0 2024-03-26T17:59:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f88854ef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8880100150 0x7f888019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f88854ef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8880100150 0x7f888019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41306/0 (socket says 172.21.15.49:41306) 2024-03-26T17:59:59.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f88854ef640 1 -- 172.21.15.49:0/3916769247 learned_addr learned my addr 172.21.15.49:0/3916769247 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f8885cf0640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8880105470 0x7f88801a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.621+0000 7f8884cee640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8880102ac0 0x7f888019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f88854ef640 1 -- 172.21.15.49:0/3916769247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8880102ac0 msgr2=0x7f888019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f88854ef640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8880102ac0 0x7f888019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f88854ef640 1 -- 172.21.15.49:0/3916769247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8880105470 msgr2=0x7f88801a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f88854ef640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8880105470 0x7f88801a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f88854ef640 1 -- 172.21.15.49:0/3916769247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88801a5d70 con 0x7f8880100150 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f8885cf0640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8880105470 0x7f88801a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f88854ef640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8880100150 0x7f888019e150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f887800d950 tx=0x7f887800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f88767fc640 1 -- 172.21.15.49:0/3916769247 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878017070 con 0x7f8880100150 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f88767fc640 1 -- 172.21.15.49:0/3916769247 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8878009c30 con 0x7f8880100150 2024-03-26T17:59:59.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f88767fc640 1 -- 172.21.15.49:0/3916769247 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887800f6b0 con 0x7f8880100150 2024-03-26T17:59:59.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f8887753640 1 -- 172.21.15.49:0/3916769247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88801a6060 con 0x7f8880100150 2024-03-26T17:59:59.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.622+0000 7f8887753640 1 -- 172.21.15.49:0/3916769247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88801a6560 con 0x7f8880100150 2024-03-26T17:59:59.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.623+0000 7f8887753640 1 -- 172.21.15.49:0/3916769247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88800fd860 con 0x7f8880100150 2024-03-26T17:59:59.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.624+0000 7f88767fc640 1 -- 172.21.15.49:0/3916769247 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f88780109d0 con 0x7f8880100150 2024-03-26T17:59:59.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.624+0000 7f88767fc640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f886c0a2f20 0x7f886c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.625+0000 7f8884cee640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f886c0a2f20 0x7f886c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.625+0000 7f88767fc640 1 -- 172.21.15.49:0/3916769247 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f88780cf200 con 0x7f8880100150 2024-03-26T17:59:59.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.625+0000 7f8884cee640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f886c0a2f20 0x7f886c0a53e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88700095f0 tx=0x7f8870005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.629+0000 7f88767fc640 1 -- 172.21.15.49:0/3916769247 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8878009de0 con 0x7f8880100150 2024-03-26T17:59:59.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.813+0000 7f8887753640 1 -- 172.21.15.49:0/3916769247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f88800624f0 con 0x7f8880100150 2024-03-26T17:59:59.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.813+0000 7f88767fc640 1 -- 172.21.15.49:0/3916769247 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f887809b7a0 con 0x7f8880100150 2024-03-26T17:59:59.796 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 -- 172.21.15.49:0/3916769247 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f886c0a2f20 msgr2=0x7f886c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f886c0a2f20 0x7f886c0a53e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f88700095f0 tx=0x7f8870005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 -- 172.21.15.49:0/3916769247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8880100150 msgr2=0x7f888019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8880100150 0x7f888019e150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f887800d950 tx=0x7f887800de10 comp rx=0 tx=0).stop 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 -- 172.21.15.49:0/3916769247 shutdown_connections 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f886c0a2f20 0x7f886c0a53e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8880102ac0 0x7f888019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8880105470 0x7f88801a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 --2- 172.21.15.49:0/3916769247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8880100150 0x7f888019e150 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 -- 172.21.15.49:0/3916769247 >> 172.21.15.49:0/3916769247 conn(0x7f88800fbd70 msgr2=0x7f8880100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 -- 172.21.15.49:0/3916769247 shutdown_connections 2024-03-26T17:59:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.815+0000 7f8887753640 1 -- 172.21.15.49:0/3916769247 wait complete. 2024-03-26T17:59:59.809 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T17:59:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.937+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/488589605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d38102b70 msgr2=0x7f9d38102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.937+0000 7f9d3d7c4640 1 --2- 172.21.15.49:0/488589605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d38102b70 0x7f9d38102ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009860 tx=0x7f9d2002bb70 comp rx=0 tx=0).stop 2024-03-26T17:59:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.937+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/488589605 shutdown_connections 2024-03-26T17:59:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.937+0000 7f9d3d7c4640 1 --2- 172.21.15.49:0/488589605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d38103530 0x7f9d38109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.937+0000 7f9d3d7c4640 1 --2- 172.21.15.49:0/488589605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d38102b70 0x7f9d38102ff0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.937+0000 7f9d3d7c4640 1 --2- 172.21.15.49:0/488589605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d380ffba0 0x7f9d380fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.937+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/488589605 >> 172.21.15.49:0/488589605 conn(0x7f9d380fb4b0 msgr2=0x7f9d380fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.937+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/488589605 shutdown_connections 2024-03-26T17:59:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.937+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/488589605 wait complete. 2024-03-26T17:59:59.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d3d7c4640 1 Processor -- start 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d3d7c4640 1 -- start start 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d3d7c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d380ffba0 0x7f9d38197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d3d7c4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d38102b70 0x7f9d381980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d3d7c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d38103530 0x7f9d3819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d3d7c4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3810d580 con 0x7f9d38103530 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d3d7c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3810d400 con 0x7f9d380ffba0 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d3d7c4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3810d700 con 0x7f9d38102b70 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d377fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d38103530 0x7f9d3819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d377fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d38103530 0x7f9d3819f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41322/0 (socket says 172.21.15.49:41322) 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d36ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d380ffba0 0x7f9d38197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d377fe640 1 -- 172.21.15.49:0/2853349732 learned_addr learned my addr 172.21.15.49:0/2853349732 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d367fc640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d38102b70 0x7f9d381980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d377fe640 1 -- 172.21.15.49:0/2853349732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d38102b70 msgr2=0x7f9d381980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d377fe640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d38102b70 0x7f9d381980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d377fe640 1 -- 172.21.15.49:0/2853349732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d380ffba0 msgr2=0x7f9d38197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.938+0000 7f9d377fe640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d380ffba0 0x7f9d38197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.939+0000 7f9d377fe640 1 -- 172.21.15.49:0/2853349732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3819f7b0 con 0x7f9d38103530 2024-03-26T17:59:59.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.939+0000 7f9d36ffd640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d380ffba0 0x7f9d38197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.939+0000 7f9d377fe640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d38103530 0x7f9d3819f140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800eaa0 tx=0x7f9d2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.939+0000 7f9d2ffff640 1 -- 172.21.15.49:0/2853349732 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800cdb0 con 0x7f9d38103530 2024-03-26T17:59:59.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.939+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/2853349732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3819faa0 con 0x7f9d38103530 2024-03-26T17:59:59.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.939+0000 7f9d2ffff640 1 -- 172.21.15.49:0/2853349732 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d28005020 con 0x7f9d38103530 2024-03-26T17:59:59.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.939+0000 7f9d2ffff640 1 -- 172.21.15.49:0/2853349732 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d280116d0 con 0x7f9d38103530 2024-03-26T17:59:59.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.939+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/2853349732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3819ffc0 con 0x7f9d38103530 2024-03-26T17:59:59.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.940+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/2853349732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d38108ca0 con 0x7f9d38103530 2024-03-26T17:59:59.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.941+0000 7f9d2ffff640 1 -- 172.21.15.49:0/2853349732 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f9d280054d0 con 0x7f9d38103530 2024-03-26T17:59:59.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.941+0000 7f9d2ffff640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9d240a2f20 0x7f9d240a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.942+0000 7f9d36ffd640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9d240a2f20 0x7f9d240a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.942+0000 7f9d2ffff640 1 -- 172.21.15.49:0/2853349732 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9d280cf830 con 0x7f9d38103530 2024-03-26T17:59:59.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.942+0000 7f9d36ffd640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9d240a2f20 0x7f9d240a53e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d300096c0 tx=0x7f9d30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T17:59:59.946+0000 7f9d2ffff640 1 -- 172.21.15.49:0/2853349732 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9d28016030 con 0x7f9d38103530 2024-03-26T18:00:00.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.128+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/2853349732 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9d3802d290 con 0x7f9d38103530 2024-03-26T18:00:00.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.129+0000 7f9d2ffff640 1 -- 172.21.15.49:0/2853349732 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f9d2809c420 con 0x7f9d38103530 2024-03-26T18:00:00.112 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/2853349732 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9d240a2f20 msgr2=0x7f9d240a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9d240a2f20 0x7f9d240a53e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d300096c0 tx=0x7f9d30009260 comp rx=0 tx=0).stop 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/2853349732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d38103530 msgr2=0x7f9d3819f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d38103530 0x7f9d3819f140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800eaa0 tx=0x7f9d2800ef60 comp rx=0 tx=0).stop 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/2853349732 shutdown_connections 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9d240a2f20 0x7f9d240a53e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9d38102b70 0x7f9d381980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d38103530 0x7f9d3819f140 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 --2- 172.21.15.49:0/2853349732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d380ffba0 0x7f9d38197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/2853349732 >> 172.21.15.49:0/2853349732 conn(0x7f9d380fb4b0 msgr2=0x7f9d38105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:00.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/2853349732 shutdown_connections 2024-03-26T18:00:00.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.131+0000 7f9d3d7c4640 1 -- 172.21.15.49:0/2853349732 wait complete. 2024-03-26T18:00:00.125 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:00:00.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.254+0000 7f52668c0640 1 -- 172.21.15.49:0/4105337590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5260102ae0 msgr2=0x7f5260102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.254+0000 7f52668c0640 1 --2- 172.21.15.49:0/4105337590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5260102ae0 0x7f5260102f60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5254009900 tx=0x7f525402bc50 comp rx=0 tx=0).stop 2024-03-26T18:00:00.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.254+0000 7f52668c0640 1 -- 172.21.15.49:0/4105337590 shutdown_connections 2024-03-26T18:00:00.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.254+0000 7f52668c0640 1 --2- 172.21.15.49:0/4105337590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52601034a0 0x7f5260109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.254+0000 7f52668c0640 1 --2- 172.21.15.49:0/4105337590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5260102ae0 0x7f5260102f60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.254+0000 7f52668c0640 1 --2- 172.21.15.49:0/4105337590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52601018f0 0x7f5260101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.254+0000 7f52668c0640 1 -- 172.21.15.49:0/4105337590 >> 172.21.15.49:0/4105337590 conn(0x7f52600fce60 msgr2=0x7f52600ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:00.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.255+0000 7f52668c0640 1 -- 172.21.15.49:0/4105337590 shutdown_connections 2024-03-26T18:00:00.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.255+0000 7f52668c0640 1 -- 172.21.15.49:0/4105337590 wait complete. 2024-03-26T18:00:00.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.255+0000 7f52668c0640 1 Processor -- start 2024-03-26T18:00:00.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.255+0000 7f52668c0640 1 -- start start 2024-03-26T18:00:00.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.255+0000 7f52668c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52601018f0 0x7f5260197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.255+0000 7f52668c0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5260102ae0 0x7f52601980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.255+0000 7f52668c0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52601034a0 0x7f526019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.255+0000 7f52668c0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f526010d5a0 con 0x7f52601018f0 2024-03-26T18:00:00.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.255+0000 7f52668c0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f526010d420 con 0x7f52601034a0 2024-03-26T18:00:00.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.255+0000 7f52668c0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f526010d720 con 0x7f5260102ae0 2024-03-26T18:00:00.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52601018f0 0x7f5260197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52601018f0 0x7f5260197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41328/0 (socket says 172.21.15.49:41328) 2024-03-26T18:00:00.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525ffff640 1 -- 172.21.15.49:0/835177725 learned_addr learned my addr 172.21.15.49:0/835177725 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:00.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f5264e5d640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52601034a0 0x7f526019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525f7fe640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5260102ae0 0x7f52601980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525ffff640 1 -- 172.21.15.49:0/835177725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5260102ae0 msgr2=0x7f52601980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525ffff640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5260102ae0 0x7f52601980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525ffff640 1 -- 172.21.15.49:0/835177725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52601034a0 msgr2=0x7f526019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525ffff640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52601034a0 0x7f526019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525ffff640 1 -- 172.21.15.49:0/835177725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526019f7d0 con 0x7f52601018f0 2024-03-26T18:00:00.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f5264e5d640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52601034a0 0x7f526019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:00.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525ffff640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52601018f0 0x7f5260197bb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f525000e9c0 tx=0x7f525000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:00.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525d7fa640 1 -- 172.21.15.49:0/835177725 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525000cde0 con 0x7f52601018f0 2024-03-26T18:00:00.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525d7fa640 1 -- 172.21.15.49:0/835177725 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5250010910 con 0x7f52601018f0 2024-03-26T18:00:00.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f52668c0640 1 -- 172.21.15.49:0/835177725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526019fac0 con 0x7f52601018f0 2024-03-26T18:00:00.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f525d7fa640 1 -- 172.21.15.49:0/835177725 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250018a00 con 0x7f52601018f0 2024-03-26T18:00:00.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.256+0000 7f52668c0640 1 -- 172.21.15.49:0/835177725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526019ffe0 con 0x7f52601018f0 2024-03-26T18:00:00.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.257+0000 7f52668c0640 1 -- 172.21.15.49:0/835177725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5260108bb0 con 0x7f52601018f0 2024-03-26T18:00:00.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.259+0000 7f525d7fa640 1 -- 172.21.15.49:0/835177725 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f525001f020 con 0x7f52601018f0 2024-03-26T18:00:00.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.260+0000 7f525d7fa640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52480a2f20 0x7f52480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.260+0000 7f525d7fa640 1 -- 172.21.15.49:0/835177725 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f52500cfea0 con 0x7f52601018f0 2024-03-26T18:00:00.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.260+0000 7f525f7fe640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52480a2f20 0x7f52480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.260+0000 7f525f7fe640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52480a2f20 0x7f52480a53e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f525402c260 tx=0x7f5254005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:00.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.263+0000 7f525d7fa640 1 -- 172.21.15.49:0/835177725 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f525009ca90 con 0x7f52601018f0 2024-03-26T18:00:00.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.461+0000 7f52668c0640 1 -- 172.21.15.49:0/835177725 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f526002d3f0 con 0x7f52601018f0 2024-03-26T18:00:00.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.461+0000 7f525d7fa640 1 -- 172.21.15.49:0/835177725 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f525009c390 con 0x7f52601018f0 2024-03-26T18:00:00.445 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-26T18:00:00.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 -- 172.21.15.49:0/835177725 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52480a2f20 msgr2=0x7f52480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52480a2f20 0x7f52480a53e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f525402c260 tx=0x7f5254005fd0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 -- 172.21.15.49:0/835177725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52601018f0 msgr2=0x7f5260197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52601018f0 0x7f5260197bb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f525000e9c0 tx=0x7f525000ee80 comp rx=0 tx=0).stop 2024-03-26T18:00:00.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 -- 172.21.15.49:0/835177725 shutdown_connections 2024-03-26T18:00:00.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f52480a2f20 0x7f52480a53e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5260102ae0 0x7f52601980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52601034a0 0x7f526019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 --2- 172.21.15.49:0/835177725 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52601018f0 0x7f5260197bb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 -- 172.21.15.49:0/835177725 >> 172.21.15.49:0/835177725 conn(0x7f52600fce60 msgr2=0x7f5260105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:00.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.463+0000 7f52668c0640 1 -- 172.21.15.49:0/835177725 shutdown_connections 2024-03-26T18:00:00.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.464+0000 7f52668c0640 1 -- 172.21.15.49:0/835177725 wait complete. 2024-03-26T18:00:00.458 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T18:00:00.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.586+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/1464428283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cbc1075c0 msgr2=0x7f2cbc109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.586+0000 7f2cc1e4b640 1 --2- 172.21.15.49:0/1464428283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cbc1075c0 0x7f2cbc109a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009be0 tx=0x7f2cb402bef0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.586+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/1464428283 shutdown_connections 2024-03-26T18:00:00.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.586+0000 7f2cc1e4b640 1 --2- 172.21.15.49:0/1464428283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cbc104c90 0x7f2cbc107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 --2- 172.21.15.49:0/1464428283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cbc1075c0 0x7f2cbc109a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 --2- 172.21.15.49:0/1464428283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc102360 0x7f2cbc104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/1464428283 >> 172.21.15.49:0/1464428283 conn(0x7f2cbc0fbd70 msgr2=0x7f2cbc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/1464428283 shutdown_connections 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/1464428283 wait complete. 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 Processor -- start 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 -- start start 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cbc102360 0x7f2cbc101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc104c90 0x7f2cbc100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cbc1075c0 0x7f2cbc100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10d5d0 con 0x7f2cbc104c90 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10d450 con 0x7f2cbc102360 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.587+0000 7f2cc1e4b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10d750 con 0x7f2cbc1075c0 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc104c90 0x7f2cbc100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cbc102360 0x7f2cbc101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cbc102360 0x7f2cbc101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43244/0 (socket says 172.21.15.49:43244) 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc104c90 0x7f2cbc100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41346/0 (socket says 172.21.15.49:41346) 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbb7fe640 1 -- 172.21.15.49:0/207282982 learned_addr learned my addr 172.21.15.49:0/207282982 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbbfff640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cbc1075c0 0x7f2cbc100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbb7fe640 1 -- 172.21.15.49:0/207282982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cbc1075c0 msgr2=0x7f2cbc100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbb7fe640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cbc1075c0 0x7f2cbc100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbb7fe640 1 -- 172.21.15.49:0/207282982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc104c90 msgr2=0x7f2cbc100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbb7fe640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc104c90 0x7f2cbc100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbb7fe640 1 -- 172.21.15.49:0/207282982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc1010e0 con 0x7f2cbc102360 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbaffd640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc104c90 0x7f2cbc100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cbb7fe640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cbc102360 0x7f2cbc101cd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00e9c0 tx=0x7f2cac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cb8ff9640 1 -- 172.21.15.49:0/207282982 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00cde0 con 0x7f2cbc102360 2024-03-26T18:00:00.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cb8ff9640 1 -- 172.21.15.49:0/207282982 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cac0035f0 con 0x7f2cbc102360 2024-03-26T18:00:00.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/207282982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc1a1cc0 con 0x7f2cbc102360 2024-03-26T18:00:00.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cb8ff9640 1 -- 172.21.15.49:0/207282982 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac010690 con 0x7f2cbc102360 2024-03-26T18:00:00.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.588+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/207282982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc1a21b0 con 0x7f2cbc102360 2024-03-26T18:00:00.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.589+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/207282982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc0fd860 con 0x7f2cbc102360 2024-03-26T18:00:00.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.590+0000 7f2cb8ff9640 1 -- 172.21.15.49:0/207282982 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f2cac0037a0 con 0x7f2cbc102360 2024-03-26T18:00:00.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.591+0000 7f2cb8ff9640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ca40a2fa0 0x7f2ca40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.591+0000 7f2cbaffd640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ca40a2fa0 0x7f2ca40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.591+0000 7f2cb8ff9640 1 -- 172.21.15.49:0/207282982 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2cac0cf940 con 0x7f2cbc102360 2024-03-26T18:00:00.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.591+0000 7f2cbaffd640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ca40a2fa0 0x7f2ca40a5460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc101a40 tx=0x7f2cb000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:00.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.595+0000 7f2cb8ff9640 1 -- 172.21.15.49:0/207282982 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2cac09c530 con 0x7f2cbc102360 2024-03-26T18:00:00.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:00 smithi115 ceph-mon[36174]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-03-26T18:00:00.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:00 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/670015112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:00.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:00 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3916769247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:00:00.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:00 smithi115 ceph-mon[36174]: overall HEALTH_OK 2024-03-26T18:00:00.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:00 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2853349732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:00:00.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:00 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/835177725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:00:00.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:00 smithi049 ceph-mon[32331]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-03-26T18:00:00.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:00 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/670015112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:00.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:00 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3916769247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:00:00.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:00 smithi049 ceph-mon[32331]: overall HEALTH_OK 2024-03-26T18:00:00.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:00 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2853349732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:00:00.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:00 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/835177725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:00:00.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.778+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/207282982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2cbc0624c0 con 0x7f2cbc102360 2024-03-26T18:00:00.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.778+0000 7f2cb8ff9640 1 -- 172.21.15.49:0/207282982 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f2cac09be30 con 0x7f2cbc102360 2024-03-26T18:00:00.762 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:00:00.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.780+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/207282982 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ca40a2fa0 msgr2=0x7f2ca40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.780+0000 7f2cc1e4b640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ca40a2fa0 0x7f2ca40a5460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc101a40 tx=0x7f2cb000b390 comp rx=0 tx=0).stop 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.780+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/207282982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cbc102360 msgr2=0x7f2cbc101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.780+0000 7f2cc1e4b640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cbc102360 0x7f2cbc101cd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00e9c0 tx=0x7f2cac00ee80 comp rx=0 tx=0).stop 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.780+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/207282982 shutdown_connections 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.780+0000 7f2cc1e4b640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ca40a2fa0 0x7f2ca40a5460 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.780+0000 7f2cc1e4b640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cbc1075c0 0x7f2cbc100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.780+0000 7f2cc1e4b640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cbc104c90 0x7f2cbc100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.780+0000 7f2cc1e4b640 1 --2- 172.21.15.49:0/207282982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cbc102360 0x7f2cbc101cd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.780+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/207282982 >> 172.21.15.49:0/207282982 conn(0x7f2cbc0fbd70 msgr2=0x7f2cbc102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.781+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/207282982 shutdown_connections 2024-03-26T18:00:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.781+0000 7f2cc1e4b640 1 -- 172.21.15.49:0/207282982 wait complete. 2024-03-26T18:00:00.775 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T18:00:00.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:00 smithi171 ceph-mon[36033]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-03-26T18:00:00.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:00 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/670015112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:00.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:00 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3916769247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:00:00.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:00 smithi171 ceph-mon[36033]: overall HEALTH_OK 2024-03-26T18:00:00.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:00 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2853349732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:00:00.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:00 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/835177725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:00:00.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.901+0000 7fe3d1990640 1 -- 172.21.15.49:0/1640320109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc1035f0 msgr2=0x7fe3cc109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.901+0000 7fe3d1990640 1 --2- 172.21.15.49:0/1640320109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc1035f0 0x7fe3cc109e80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009cd0 tx=0x7fe3c402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.901+0000 7fe3d1990640 1 -- 172.21.15.49:0/1640320109 shutdown_connections 2024-03-26T18:00:00.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.901+0000 7fe3d1990640 1 --2- 172.21.15.49:0/1640320109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc1035f0 0x7fe3cc109e80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.901+0000 7fe3d1990640 1 --2- 172.21.15.49:0/1640320109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc102c30 0x7fe3cc1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.901+0000 7fe3d1990640 1 --2- 172.21.15.49:0/1640320109 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3cc0ffbc0 0x7fe3cc0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.901+0000 7fe3d1990640 1 -- 172.21.15.49:0/1640320109 >> 172.21.15.49:0/1640320109 conn(0x7fe3cc0fb4b0 msgr2=0x7fe3cc0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.901+0000 7fe3d1990640 1 -- 172.21.15.49:0/1640320109 shutdown_connections 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.901+0000 7fe3d1990640 1 -- 172.21.15.49:0/1640320109 wait complete. 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.901+0000 7fe3d1990640 1 Processor -- start 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3d1990640 1 -- start start 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3d1990640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc0ffbc0 0x7fe3cc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3d1990640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc102c30 0x7fe3cc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3d1990640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3cc1035f0 0x7fe3cc19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3d1990640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc10d540 con 0x7fe3cc0ffbc0 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3d1990640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc10d3c0 con 0x7fe3cc102c30 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3d1990640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc10d6c0 con 0x7fe3cc1035f0 2024-03-26T18:00:00.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3caffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc0ffbc0 0x7fe3cc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3ca7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc102c30 0x7fe3cc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3ca7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc102c30 0x7fe3cc1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43272/0 (socket says 172.21.15.49:43272) 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3caffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc0ffbc0 0x7fe3cc197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41358/0 (socket says 172.21.15.49:41358) 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3ca7fc640 1 -- 172.21.15.49:0/1997814976 learned_addr learned my addr 172.21.15.49:0/1997814976 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3cb7fe640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3cc1035f0 0x7fe3cc19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3ca7fc640 1 -- 172.21.15.49:0/1997814976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3cc1035f0 msgr2=0x7fe3cc19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3ca7fc640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3cc1035f0 0x7fe3cc19f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3ca7fc640 1 -- 172.21.15.49:0/1997814976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc0ffbc0 msgr2=0x7fe3cc197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3ca7fc640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc0ffbc0 0x7fe3cc197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3ca7fc640 1 -- 172.21.15.49:0/1997814976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3cc19f7d0 con 0x7fe3cc102c30 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3caffd640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc0ffbc0 0x7fe3cc197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.902+0000 7fe3ca7fc640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc102c30 0x7fe3cc1980f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00e9f0 tx=0x7fe3bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:00.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.903+0000 7fe3d098e640 1 -- 172.21.15.49:0/1997814976 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc00cd80 con 0x7fe3cc102c30 2024-03-26T18:00:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.903+0000 7fe3d098e640 1 -- 172.21.15.49:0/1997814976 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3bc010910 con 0x7fe3cc102c30 2024-03-26T18:00:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.903+0000 7fe3d1990640 1 -- 172.21.15.49:0/1997814976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3cc19fac0 con 0x7fe3cc102c30 2024-03-26T18:00:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.903+0000 7fe3d098e640 1 -- 172.21.15.49:0/1997814976 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc018990 con 0x7fe3cc102c30 2024-03-26T18:00:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.903+0000 7fe3d1990640 1 -- 172.21.15.49:0/1997814976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3cc1a0010 con 0x7fe3cc102c30 2024-03-26T18:00:00.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.904+0000 7fe3d1990640 1 -- 172.21.15.49:0/1997814976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3cc108d80 con 0x7fe3cc102c30 2024-03-26T18:00:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.905+0000 7fe3d098e640 1 -- 172.21.15.49:0/1997814976 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fe3bc010ac0 con 0x7fe3cc102c30 2024-03-26T18:00:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.906+0000 7fe3d098e640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe3b40a2f20 0x7fe3b40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.906+0000 7fe3caffd640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe3b40a2f20 0x7fe3b40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.906+0000 7fe3d098e640 1 -- 172.21.15.49:0/1997814976 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe3bc0cfe10 con 0x7fe3cc102c30 2024-03-26T18:00:00.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.906+0000 7fe3caffd640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe3b40a2f20 0x7fe3b40a53e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005fd0 tx=0x7fe3c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:00.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:00.909+0000 7fe3d098e640 1 -- 172.21.15.49:0/1997814976 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe3bc09c3b0 con 0x7fe3cc102c30 2024-03-26T18:00:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.105+0000 7fe3d1990640 1 -- 172.21.15.49:0/1997814976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3cc02d460 con 0x7fe3cc102c30 2024-03-26T18:00:01.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.106+0000 7fe3d098e640 1 -- 172.21.15.49:0/1997814976 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+586 (secure 0 0 0) 0x7fe3bc0a0450 con 0x7fe3cc102c30 2024-03-26T18:00:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/217046874 2024-03-27T17:52:42.621576+0000 2024-03-26T18:00:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/442612048 2024-03-27T17:52:42.621576+0000 2024-03-26T18:00:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1502160114 2024-03-27T17:52:42.621576+0000 2024-03-26T18:00:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/4249557516 2024-03-27T17:52:42.621576+0000 2024-03-26T18:00:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2824939577 2024-03-27T17:52:15.448636+0000 2024-03-26T18:00:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/48778628 2024-03-27T17:52:15.448636+0000 2024-03-26T18:00:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1915951880 2024-03-27T17:52:15.448636+0000 2024-03-26T18:00:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/48778628 2024-03-27T17:52:15.448636+0000 2024-03-26T18:00:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1560035061 2024-03-27T17:52:15.448636+0000 2024-03-26T18:00:01.090 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/4249557516 2024-03-27T17:52:42.621576+0000 2024-03-26T18:00:01.091 INFO:teuthology.orchestra.run.smithi049.stderr:listed 10 entries 2024-03-26T18:00:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 -- 172.21.15.49:0/1997814976 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe3b40a2f20 msgr2=0x7fe3b40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe3b40a2f20 0x7fe3b40a53e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005fd0 tx=0x7fe3c000b410 comp rx=0 tx=0).stop 2024-03-26T18:00:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 -- 172.21.15.49:0/1997814976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc102c30 msgr2=0x7fe3cc1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc102c30 0x7fe3cc1980f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00e9f0 tx=0x7fe3bc00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 -- 172.21.15.49:0/1997814976 shutdown_connections 2024-03-26T18:00:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe3b40a2f20 0x7fe3b40a53e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3cc1035f0 0x7fe3cc19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3cc102c30 0x7fe3cc1980f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 --2- 172.21.15.49:0/1997814976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe3cc0ffbc0 0x7fe3cc197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 -- 172.21.15.49:0/1997814976 >> 172.21.15.49:0/1997814976 conn(0x7fe3cc0fb4b0 msgr2=0x7fe3cc105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 -- 172.21.15.49:0/1997814976 shutdown_connections 2024-03-26T18:00:01.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.108+0000 7fe3d1990640 1 -- 172.21.15.49:0/1997814976 wait complete. 2024-03-26T18:00:01.103 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T18:00:01.103 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T18:00:01.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:00:01.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.284+0000 7fe0454ae640 1 -- 172.21.15.49:0/3479679162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0401023e0 msgr2=0x7fe0401047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.284+0000 7fe0454ae640 1 --2- 172.21.15.49:0/3479679162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0401023e0 0x7fe0401047d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe030009900 tx=0x7fe03002bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:01.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.284+0000 7fe0454ae640 1 -- 172.21.15.49:0/3479679162 shutdown_connections 2024-03-26T18:00:01.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.284+0000 7fe0454ae640 1 --2- 172.21.15.49:0/3479679162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040104d10 0x7fe040107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.284+0000 7fe0454ae640 1 --2- 172.21.15.49:0/3479679162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe040107640 0x7fe040109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.284+0000 7fe0454ae640 1 --2- 172.21.15.49:0/3479679162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0401023e0 0x7fe0401047d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.284+0000 7fe0454ae640 1 -- 172.21.15.49:0/3479679162 >> 172.21.15.49:0/3479679162 conn(0x7fe0400fbdf0 msgr2=0x7fe0400fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:01.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.284+0000 7fe0454ae640 1 -- 172.21.15.49:0/3479679162 shutdown_connections 2024-03-26T18:00:01.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.284+0000 7fe0454ae640 1 -- 172.21.15.49:0/3479679162 wait complete. 2024-03-26T18:00:01.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe0454ae640 1 Processor -- start 2024-03-26T18:00:01.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe0454ae640 1 -- start start 2024-03-26T18:00:01.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe0454ae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0401023e0 0x7fe040197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe0454ae640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040104d10 0x7fe040198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe0454ae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe040107640 0x7fe04019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe0454ae640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe04010d570 con 0x7fe0401023e0 2024-03-26T18:00:01.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe0454ae640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe04010d3f0 con 0x7fe040107640 2024-03-26T18:00:01.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe0454ae640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe04010d6f0 con 0x7fe040104d10 2024-03-26T18:00:01.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe03effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0401023e0 0x7fe040197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe03effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0401023e0 0x7fe040197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41380/0 (socket says 172.21.15.49:41380) 2024-03-26T18:00:01.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe03effd640 1 -- 172.21.15.49:0/630309320 learned_addr learned my addr 172.21.15.49:0/630309320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:01.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe03f7fe640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe040107640 0x7fe04019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe037fff640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040104d10 0x7fe040198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe03effd640 1 -- 172.21.15.49:0/630309320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040104d10 msgr2=0x7fe040198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe03effd640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040104d10 0x7fe040198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe03effd640 1 -- 172.21.15.49:0/630309320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe040107640 msgr2=0x7fe04019f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.285+0000 7fe03effd640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe040107640 0x7fe04019f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.286+0000 7fe03effd640 1 -- 172.21.15.49:0/630309320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04019f850 con 0x7fe0401023e0 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.286+0000 7fe037fff640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040104d10 0x7fe040198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.286+0000 7fe03effd640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0401023e0 0x7fe040197c40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe030009230 tx=0x7fe030005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.286+0000 7fe03cff9640 1 -- 172.21.15.49:0/630309320 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030041070 con 0x7fe0401023e0 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.286+0000 7fe0454ae640 1 -- 172.21.15.49:0/630309320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04019fae0 con 0x7fe0401023e0 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.286+0000 7fe03cff9640 1 -- 172.21.15.49:0/630309320 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0300044d0 con 0x7fe0401023e0 2024-03-26T18:00:01.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.286+0000 7fe03cff9640 1 -- 172.21.15.49:0/630309320 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030034440 con 0x7fe0401023e0 2024-03-26T18:00:01.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.286+0000 7fe0454ae640 1 -- 172.21.15.49:0/630309320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe04019ffd0 con 0x7fe0401023e0 2024-03-26T18:00:01.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.287+0000 7fe0454ae640 1 -- 172.21.15.49:0/630309320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0400fd8e0 con 0x7fe0401023e0 2024-03-26T18:00:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.289+0000 7fe03cff9640 1 -- 172.21.15.49:0/630309320 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fe030044070 con 0x7fe0401023e0 2024-03-26T18:00:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.289+0000 7fe03cff9640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe02c0a2f20 0x7fe02c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.289+0000 7fe03cff9640 1 -- 172.21.15.49:0/630309320 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe0300f16e0 con 0x7fe0401023e0 2024-03-26T18:00:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.289+0000 7fe037fff640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe02c0a2f20 0x7fe02c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.290+0000 7fe037fff640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe02c0a2f20 0x7fe02c0a53e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe028005d00 tx=0x7fe028005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:01.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.293+0000 7fe03cff9640 1 -- 172.21.15.49:0/630309320 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe0300be2d0 con 0x7fe0401023e0 2024-03-26T18:00:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.481+0000 7fe0454ae640 1 -- 172.21.15.49:0/630309320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe04002d410 con 0x7fe0401023e0 2024-03-26T18:00:01.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.483+0000 7fe03cff9640 1 -- 172.21.15.49:0/630309320 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24136 (secure 0 0 0) 0x7fe0300bdbd0 con 0x7fe0401023e0 2024-03-26T18:00:01.467 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:00:01.467 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:51.836345+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:00:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 -- 172.21.15.49:0/630309320 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe02c0a2f20 msgr2=0x7fe02c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe02c0a2f20 0x7fe02c0a53e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe028005d00 tx=0x7fe028005ed0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 -- 172.21.15.49:0/630309320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0401023e0 msgr2=0x7fe040197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0401023e0 0x7fe040197c40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe030009230 tx=0x7fe030005d50 comp rx=0 tx=0).stop 2024-03-26T18:00:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 -- 172.21.15.49:0/630309320 shutdown_connections 2024-03-26T18:00:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe02c0a2f20 0x7fe02c0a53e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe040104d10 0x7fe040198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe040107640 0x7fe04019f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 --2- 172.21.15.49:0/630309320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe0401023e0 0x7fe040197c40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 -- 172.21.15.49:0/630309320 >> 172.21.15.49:0/630309320 conn(0x7fe0400fbdf0 msgr2=0x7fe040102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 -- 172.21.15.49:0/630309320 shutdown_connections 2024-03-26T18:00:01.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.485+0000 7fe0454ae640 1 -- 172.21.15.49:0/630309320 wait complete. 2024-03-26T18:00:01.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:59:45.382542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:59:46.221139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T17:59:49.890383+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T18:00:01.480 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:00:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.608+0000 7f67c1589640 1 -- 172.21.15.49:0/2669703650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67bc1018f0 msgr2=0x7f67bc101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.608+0000 7f67c1589640 1 --2- 172.21.15.49:0/2669703650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67bc1018f0 0x7f67bc101d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009900 tx=0x7f67b402bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 -- 172.21.15.49:0/2669703650 shutdown_connections 2024-03-26T18:00:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 --2- 172.21.15.49:0/2669703650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67bc102ae0 0x7f67bc102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 --2- 172.21.15.49:0/2669703650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67bc1034a0 0x7f67bc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 --2- 172.21.15.49:0/2669703650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67bc1018f0 0x7f67bc101d10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 -- 172.21.15.49:0/2669703650 >> 172.21.15.49:0/2669703650 conn(0x7f67bc0fce60 msgr2=0x7f67bc0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 -- 172.21.15.49:0/2669703650 shutdown_connections 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 -- 172.21.15.49:0/2669703650 wait complete. 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 Processor -- start 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 -- start start 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67bc1018f0 0x7f67bc199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67bc102ae0 0x7f67bc19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67bc1034a0 0x7f67bc1a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10d5c0 con 0x7f67bc1018f0 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10d440 con 0x7f67bc102ae0 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.609+0000 7f67c1589640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10d740 con 0x7f67bc1034a0 2024-03-26T18:00:01.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67baffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67bc1018f0 0x7f67bc199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67baffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67bc1018f0 0x7f67bc199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41396/0 (socket says 172.21.15.49:41396) 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67baffd640 1 -- 172.21.15.49:0/2388636570 learned_addr learned my addr 172.21.15.49:0/2388636570 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67ba7fc640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67bc102ae0 0x7f67bc19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67baffd640 1 -- 172.21.15.49:0/2388636570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67bc1034a0 msgr2=0x7f67bc1a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67baffd640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67bc1034a0 0x7f67bc1a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67baffd640 1 -- 172.21.15.49:0/2388636570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67bc102ae0 msgr2=0x7f67bc19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67baffd640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67bc102ae0 0x7f67bc19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67baffd640 1 -- 172.21.15.49:0/2388636570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc1a1aa0 con 0x7f67bc1018f0 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67baffd640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67bc1018f0 0x7f67bc199ed0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f67b4000c00 tx=0x7f67b4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67b3fff640 1 -- 172.21.15.49:0/2388636570 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4041070 con 0x7f67bc1018f0 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67b3fff640 1 -- 172.21.15.49:0/2388636570 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b40044d0 con 0x7f67bc1018f0 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67c1589640 1 -- 172.21.15.49:0/2388636570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc1a1d30 con 0x7f67bc1018f0 2024-03-26T18:00:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67b3fff640 1 -- 172.21.15.49:0/2388636570 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4034440 con 0x7f67bc1018f0 2024-03-26T18:00:01.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.610+0000 7f67c1589640 1 -- 172.21.15.49:0/2388636570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a21d0 con 0x7f67bc1018f0 2024-03-26T18:00:01.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.611+0000 7f67c1589640 1 -- 172.21.15.49:0/2388636570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67bc1a2400 con 0x7f67bc1018f0 2024-03-26T18:00:01.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.612+0000 7f67b3fff640 1 -- 172.21.15.49:0/2388636570 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f67b4044070 con 0x7f67bc1018f0 2024-03-26T18:00:01.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.613+0000 7f67b3fff640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67a80a2f20 0x7f67a80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.613+0000 7f67ba7fc640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67a80a2f20 0x7f67a80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.613+0000 7f67b3fff640 1 -- 172.21.15.49:0/2388636570 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f67b40f11e0 con 0x7f67bc1018f0 2024-03-26T18:00:01.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.613+0000 7f67ba7fc640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67a80a2f20 0x7f67a80a53e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67bc19b390 tx=0x7f67a4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:01.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.618+0000 7f67b3fff640 1 -- 172.21.15.49:0/2388636570 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f67b40bde70 con 0x7f67bc1018f0 2024-03-26T18:00:01.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:01 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/207282982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:00:01.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:01 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1997814976' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:00:01.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:01 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/207282982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:00:01.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:01 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1997814976' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:00:01.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.800+0000 7f67c1589640 1 -- 172.21.15.49:0/2388636570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f67bc1a25d0 con 0x7f67bc1018f0 2024-03-26T18:00:01.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.801+0000 7f67b3fff640 1 -- 172.21.15.49:0/2388636570 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f67b40bd770 con 0x7f67bc1018f0 2024-03-26T18:00:01.784 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 -- 172.21.15.49:0/2388636570 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67a80a2f20 msgr2=0x7f67a80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67a80a2f20 0x7f67a80a53e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67bc19b390 tx=0x7f67a4009110 comp rx=0 tx=0).stop 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 -- 172.21.15.49:0/2388636570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67bc1018f0 msgr2=0x7f67bc199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67bc1018f0 0x7f67bc199ed0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f67b4000c00 tx=0x7f67b4005d50 comp rx=0 tx=0).stop 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 -- 172.21.15.49:0/2388636570 shutdown_connections 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67a80a2f20 0x7f67a80a53e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67bc1034a0 0x7f67bc1a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67bc102ae0 0x7f67bc19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 --2- 172.21.15.49:0/2388636570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67bc1018f0 0x7f67bc199ed0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 -- 172.21.15.49:0/2388636570 >> 172.21.15.49:0/2388636570 conn(0x7f67bc0fce60 msgr2=0x7f67bc073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 -- 172.21.15.49:0/2388636570 shutdown_connections 2024-03-26T18:00:01.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.803+0000 7f67c1589640 1 -- 172.21.15.49:0/2388636570 wait complete. 2024-03-26T18:00:01.798 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:00:01.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:01 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/207282982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:00:01.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:01 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1997814976' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:00:01.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 -- 172.21.15.49:0/1760563355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84841018d0 msgr2=0x7f8484101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 --2- 172.21.15.49:0/1760563355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84841018d0 0x7f8484101cf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8474009900 tx=0x7f847402bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:01.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 -- 172.21.15.49:0/1760563355 shutdown_connections 2024-03-26T18:00:01.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 --2- 172.21.15.49:0/1760563355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8484103480 0x7f8484109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 --2- 172.21.15.49:0/1760563355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8484102ac0 0x7f8484102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 --2- 172.21.15.49:0/1760563355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84841018d0 0x7f8484101cf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 -- 172.21.15.49:0/1760563355 >> 172.21.15.49:0/1760563355 conn(0x7f84840fce60 msgr2=0x7f84840ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:01.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 -- 172.21.15.49:0/1760563355 shutdown_connections 2024-03-26T18:00:01.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 -- 172.21.15.49:0/1760563355 wait complete. 2024-03-26T18:00:01.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 Processor -- start 2024-03-26T18:00:01.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.926+0000 7f848b737640 1 -- start start 2024-03-26T18:00:01.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f848b737640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84841018d0 0x7f8484075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f848b737640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8484102ac0 0x7f8484073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f848b737640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484103480 0x7f84840741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f848b737640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f848410d550 con 0x7f8484103480 2024-03-26T18:00:01.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f848b737640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f848410d3d0 con 0x7f84841018d0 2024-03-26T18:00:01.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f848b737640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f848410d6d0 con 0x7f8484102ac0 2024-03-26T18:00:01.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f8489cd4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484103480 0x7f84840741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f84894d3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84841018d0 0x7f8484075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f8489cd4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484103480 0x7f84840741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41420/0 (socket says 172.21.15.49:41420) 2024-03-26T18:00:01.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f84894d3640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84841018d0 0x7f8484075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43326/0 (socket says 172.21.15.49:43326) 2024-03-26T18:00:01.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f8489cd4640 1 -- 172.21.15.49:0/3314620484 learned_addr learned my addr 172.21.15.49:0/3314620484 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:01.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f8488cd2640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8484102ac0 0x7f8484073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f8489cd4640 1 -- 172.21.15.49:0/3314620484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8484102ac0 msgr2=0x7f8484073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f8489cd4640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8484102ac0 0x7f8484073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f8489cd4640 1 -- 172.21.15.49:0/3314620484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84841018d0 msgr2=0x7f8484075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:01.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f8489cd4640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84841018d0 0x7f8484075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:01.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f8489cd4640 1 -- 172.21.15.49:0/3314620484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8484074a80 con 0x7f8484103480 2024-03-26T18:00:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f8489cd4640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484103480 0x7f84840741f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f847c00cad0 tx=0x7f847c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f847a7fc640 1 -- 172.21.15.49:0/3314620484 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c00ae20 con 0x7f8484103480 2024-03-26T18:00:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.927+0000 7f847a7fc640 1 -- 172.21.15.49:0/3314620484 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847c005020 con 0x7f8484103480 2024-03-26T18:00:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.928+0000 7f847a7fc640 1 -- 172.21.15.49:0/3314620484 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c00f6f0 con 0x7f8484103480 2024-03-26T18:00:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.928+0000 7f848b737640 1 -- 172.21.15.49:0/3314620484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84841a1da0 con 0x7f8484103480 2024-03-26T18:00:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.928+0000 7f848b737640 1 -- 172.21.15.49:0/3314620484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84841a2260 con 0x7f8484103480 2024-03-26T18:00:01.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.928+0000 7f848b737640 1 -- 172.21.15.49:0/3314620484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8484108c10 con 0x7f8484103480 2024-03-26T18:00:01.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.930+0000 7f847a7fc640 1 -- 172.21.15.49:0/3314620484 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f847c0051d0 con 0x7f8484103480 2024-03-26T18:00:01.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.930+0000 7f847a7fc640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84700a2f20 0x7f84700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:01.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.930+0000 7f84894d3640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84700a2f20 0x7f84700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:01.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.930+0000 7f847a7fc640 1 -- 172.21.15.49:0/3314620484 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f847c0cf920 con 0x7f8484103480 2024-03-26T18:00:01.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.931+0000 7f84894d3640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84700a2f20 0x7f84700a53e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8474000c00 tx=0x7f8474005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:01.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:01.935+0000 7f847a7fc640 1 -- 172.21.15.49:0/3314620484 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f847c014030 con 0x7f8484103480 2024-03-26T18:00:02.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.118+0000 7f848b737640 1 -- 172.21.15.49:0/3314620484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f84840624c0 con 0x7f8484103480 2024-03-26T18:00:02.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.119+0000 7f847a7fc640 1 -- 172.21.15.49:0/3314620484 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f847c09bec0 con 0x7f8484103480 2024-03-26T18:00:02.102 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:00:02.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 -- 172.21.15.49:0/3314620484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84700a2f20 msgr2=0x7f84700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84700a2f20 0x7f84700a53e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8474000c00 tx=0x7f8474005fd0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 -- 172.21.15.49:0/3314620484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484103480 msgr2=0x7f84840741f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484103480 0x7f84840741f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f847c00cad0 tx=0x7f847c00cf90 comp rx=0 tx=0).stop 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 -- 172.21.15.49:0/3314620484 shutdown_connections 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84700a2f20 0x7f84700a53e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8484102ac0 0x7f8484073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8484103480 0x7f84840741f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 --2- 172.21.15.49:0/3314620484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84841018d0 0x7f8484075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 -- 172.21.15.49:0/3314620484 >> 172.21.15.49:0/3314620484 conn(0x7f84840fce60 msgr2=0x7f8484105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 -- 172.21.15.49:0/3314620484 shutdown_connections 2024-03-26T18:00:02.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.121+0000 7f848b737640 1 -- 172.21.15.49:0/3314620484 wait complete. 2024-03-26T18:00:02.115 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:00:02.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/16801623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c41075c0 msgr2=0x7fc6c4109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 --2- 172.21.15.49:0/16801623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c41075c0 0x7fc6c4109a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009ca0 tx=0x7fc6bc02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/16801623 shutdown_connections 2024-03-26T18:00:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 --2- 172.21.15.49:0/16801623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6c4104c90 0x7fc6c4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 --2- 172.21.15.49:0/16801623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c41075c0 0x7fc6c4109a10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 --2- 172.21.15.49:0/16801623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6c4102360 0x7fc6c4104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/16801623 >> 172.21.15.49:0/16801623 conn(0x7fc6c40fbd70 msgr2=0x7fc6c40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/16801623 shutdown_connections 2024-03-26T18:00:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/16801623 wait complete. 2024-03-26T18:00:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 Processor -- start 2024-03-26T18:00:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.243+0000 7fc6ca7c4640 1 -- start start 2024-03-26T18:00:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6ca7c4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6c4102360 0x7fc6c4197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6ca7c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c4104c90 0x7fc6c41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6ca7c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6c41075c0 0x7fc6c419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6ca7c4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c410d560 con 0x7fc6c4104c90 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6ca7c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c410d3e0 con 0x7fc6c41075c0 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6ca7c4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c410d6e0 con 0x7fc6c4102360 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c4104c90 0x7fc6c41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c4104c90 0x7fc6c41980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41440/0 (socket says 172.21.15.49:41440) 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c37fe640 1 -- 172.21.15.49:0/2157596503 learned_addr learned my addr 172.21.15.49:0/2157596503 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c8d61640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6c41075c0 0x7fc6c419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c3fff640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6c4102360 0x7fc6c4197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c37fe640 1 -- 172.21.15.49:0/2157596503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6c4102360 msgr2=0x7fc6c4197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c37fe640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6c4102360 0x7fc6c4197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c37fe640 1 -- 172.21.15.49:0/2157596503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6c41075c0 msgr2=0x7fc6c419f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c37fe640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6c41075c0 0x7fc6c419f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c37fe640 1 -- 172.21.15.49:0/2157596503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c419f7c0 con 0x7fc6c4104c90 2024-03-26T18:00:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c37fe640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c4104c90 0x7fc6c41980e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800d770 tx=0x7fc6b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.244+0000 7fc6c17fa640 1 -- 172.21.15.49:0/2157596503 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8017070 con 0x7fc6c4104c90 2024-03-26T18:00:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.245+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/2157596503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c419fab0 con 0x7fc6c4104c90 2024-03-26T18:00:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.245+0000 7fc6c17fa640 1 -- 172.21.15.49:0/2157596503 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6b8009c30 con 0x7fc6c4104c90 2024-03-26T18:00:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.245+0000 7fc6c17fa640 1 -- 172.21.15.49:0/2157596503 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b800f620 con 0x7fc6c4104c90 2024-03-26T18:00:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.245+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/2157596503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c41a0050 con 0x7fc6c4104c90 2024-03-26T18:00:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.246+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/2157596503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c40fd860 con 0x7fc6c4104c90 2024-03-26T18:00:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.247+0000 7fc6c17fa640 1 -- 172.21.15.49:0/2157596503 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fc6b801a070 con 0x7fc6c4104c90 2024-03-26T18:00:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.247+0000 7fc6c17fa640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6ac0a2f20 0x7fc6ac0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.247+0000 7fc6c3fff640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6ac0a2f20 0x7fc6ac0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.247+0000 7fc6c17fa640 1 -- 172.21.15.49:0/2157596503 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc6b80cf400 con 0x7fc6c4104c90 2024-03-26T18:00:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.248+0000 7fc6c3fff640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6ac0a2f20 0x7fc6ac0a53e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40095f0 tx=0x7fc6b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:02.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.252+0000 7fc6c17fa640 1 -- 172.21.15.49:0/2157596503 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc6b8009de0 con 0x7fc6c4104c90 2024-03-26T18:00:02.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.437+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/2157596503 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc6c40624c0 con 0x7fc6c4104c90 2024-03-26T18:00:02.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.438+0000 7fc6c17fa640 1 -- 172.21.15.49:0/2157596503 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fc6b809b9a0 con 0x7fc6c4104c90 2024-03-26T18:00:02.421 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-26T18:00:02.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/2157596503 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6ac0a2f20 msgr2=0x7fc6ac0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6ac0a2f20 0x7fc6ac0a53e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40095f0 tx=0x7fc6b4005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/2157596503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c4104c90 msgr2=0x7fc6c41980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c4104c90 0x7fc6c41980e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800d770 tx=0x7fc6b800dc30 comp rx=0 tx=0).stop 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/2157596503 shutdown_connections 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6ac0a2f20 0x7fc6ac0a53e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6c41075c0 0x7fc6c419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc6c4104c90 0x7fc6c41980e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 --2- 172.21.15.49:0/2157596503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6c4102360 0x7fc6c4197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/2157596503 >> 172.21.15.49:0/2157596503 conn(0x7fc6c40fbd70 msgr2=0x7fc6c4102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/2157596503 shutdown_connections 2024-03-26T18:00:02.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.440+0000 7fc6ca7c4640 1 -- 172.21.15.49:0/2157596503 wait complete. 2024-03-26T18:00:02.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T18:00:02.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 -- 172.21.15.49:0/3681471702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facfc102370 msgr2=0x7facfc108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 --2- 172.21.15.49:0/3681471702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facfc102370 0x7facfc108c00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7facf4009ca0 tx=0x7facf402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 -- 172.21.15.49:0/3681471702 shutdown_connections 2024-03-26T18:00:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 --2- 172.21.15.49:0/3681471702 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facfc102370 0x7facfc108c00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 --2- 172.21.15.49:0/3681471702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facfc1019b0 0x7facfc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 --2- 172.21.15.49:0/3681471702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facfc1007c0 0x7facfc100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 -- 172.21.15.49:0/3681471702 >> 172.21.15.49:0/3681471702 conn(0x7facfc0fbd50 msgr2=0x7facfc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 -- 172.21.15.49:0/3681471702 shutdown_connections 2024-03-26T18:00:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 -- 172.21.15.49:0/3681471702 wait complete. 2024-03-26T18:00:02.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 Processor -- start 2024-03-26T18:00:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.563+0000 7fad02122640 1 -- start start 2024-03-26T18:00:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7fad02122640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facfc1007c0 0x7facfc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7fad02122640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facfc1019b0 0x7facfc19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7fad02122640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facfc102370 0x7facfc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7fad02122640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7facfc10c4b0 con 0x7facfc1007c0 2024-03-26T18:00:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7fad02122640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7facfc10c330 con 0x7facfc1019b0 2024-03-26T18:00:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7fad02122640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7facfc10c630 con 0x7facfc102370 2024-03-26T18:00:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facfc1007c0 0x7facfc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facfc1007c0 0x7facfc069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41468/0 (socket says 172.21.15.49:41468) 2024-03-26T18:00:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfb7fe640 1 -- 172.21.15.49:0/50420341 learned_addr learned my addr 172.21.15.49:0/50420341 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfaffd640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facfc1019b0 0x7facfc19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfbfff640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facfc102370 0x7facfc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfb7fe640 1 -- 172.21.15.49:0/50420341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facfc102370 msgr2=0x7facfc19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfb7fe640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facfc102370 0x7facfc19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfb7fe640 1 -- 172.21.15.49:0/50420341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facfc1019b0 msgr2=0x7facfc19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfb7fe640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facfc1019b0 0x7facfc19c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfb7fe640 1 -- 172.21.15.49:0/50420341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facfc19f870 con 0x7facfc1007c0 2024-03-26T18:00:02.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facfb7fe640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facfc1007c0 0x7facfc069590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7face400b450 tx=0x7face400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:02.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facf8ff9640 1 -- 172.21.15.49:0/50420341 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4015020 con 0x7facfc1007c0 2024-03-26T18:00:02.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facf8ff9640 1 -- 172.21.15.49:0/50420341 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face4011d70 con 0x7facfc1007c0 2024-03-26T18:00:02.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7facf8ff9640 1 -- 172.21.15.49:0/50420341 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face400f340 con 0x7facfc1007c0 2024-03-26T18:00:02.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7fad02122640 1 -- 172.21.15.49:0/50420341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facfc19fb00 con 0x7facfc1007c0 2024-03-26T18:00:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.564+0000 7fad02122640 1 -- 172.21.15.49:0/50420341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facfc1a0000 con 0x7facfc1007c0 2024-03-26T18:00:02.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.565+0000 7fad02122640 1 -- 172.21.15.49:0/50420341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facfc107ae0 con 0x7facfc1007c0 2024-03-26T18:00:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.566+0000 7facf8ff9640 1 -- 172.21.15.49:0/50420341 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7face4018020 con 0x7facfc1007c0 2024-03-26T18:00:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.567+0000 7facf8ff9640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7face80a2f20 0x7face80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.567+0000 7facfaffd640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7face80a2f20 0x7face80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.568+0000 7facf8ff9640 1 -- 172.21.15.49:0/50420341 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7face40cf110 con 0x7facfc1007c0 2024-03-26T18:00:02.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.568+0000 7facfaffd640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7face80a2f20 0x7face80a53e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7facfc069300 tx=0x7facec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:02.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.572+0000 7facf8ff9640 1 -- 172.21.15.49:0/50420341 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7face401f020 con 0x7facfc1007c0 2024-03-26T18:00:02.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:02 smithi115 ceph-mon[36174]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 442 B/s wr, 2 op/s 2024-03-26T18:00:02.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:02 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/630309320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:02.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:02 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2388636570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:00:02.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:02 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3314620484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:00:02.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:02 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2157596503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:00:02.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.756+0000 7fad02122640 1 -- 172.21.15.49:0/50420341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7facfc0624c0 con 0x7facfc1007c0 2024-03-26T18:00:02.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.756+0000 7facf8ff9640 1 -- 172.21.15.49:0/50420341 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7face400d040 con 0x7facfc1007c0 2024-03-26T18:00:02.740 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:00:02.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 -- 172.21.15.49:0/50420341 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7face80a2f20 msgr2=0x7face80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7face80a2f20 0x7face80a53e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7facfc069300 tx=0x7facec005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:02.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 -- 172.21.15.49:0/50420341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facfc1007c0 msgr2=0x7facfc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facfc1007c0 0x7facfc069590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7face400b450 tx=0x7face400b910 comp rx=0 tx=0).stop 2024-03-26T18:00:02.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 -- 172.21.15.49:0/50420341 shutdown_connections 2024-03-26T18:00:02.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7face80a2f20 0x7face80a53e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facfc102370 0x7facfc19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facfc1019b0 0x7facfc19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 --2- 172.21.15.49:0/50420341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7facfc1007c0 0x7facfc069590 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 -- 172.21.15.49:0/50420341 >> 172.21.15.49:0/50420341 conn(0x7facfc0fbd50 msgr2=0x7facfc104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:02.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 -- 172.21.15.49:0/50420341 shutdown_connections 2024-03-26T18:00:02.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.758+0000 7fad02122640 1 -- 172.21.15.49:0/50420341 wait complete. 2024-03-26T18:00:02.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:02 smithi049 ceph-mon[32331]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 442 B/s wr, 2 op/s 2024-03-26T18:00:02.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:02 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/630309320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:02.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:02 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2388636570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:00:02.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:02 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3314620484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:00:02.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:02 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2157596503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:00:02.753 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T18:00:02.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:02 smithi171 ceph-mon[36033]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 442 B/s wr, 2 op/s 2024-03-26T18:00:02.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:02 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/630309320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:02.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:02 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2388636570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:00:02.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:02 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3314620484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:00:02.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:02 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2157596503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:00:02.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.879+0000 7f5b154cc640 1 -- 172.21.15.49:0/3301079540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b10102370 msgr2=0x7f5b10108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.879+0000 7f5b154cc640 1 --2- 172.21.15.49:0/3301079540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b10102370 0x7f5b10108c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009be0 tx=0x7f5b0002bef0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.879+0000 7f5b154cc640 1 -- 172.21.15.49:0/3301079540 shutdown_connections 2024-03-26T18:00:02.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.879+0000 7f5b154cc640 1 --2- 172.21.15.49:0/3301079540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b101019b0 0x7f5b10101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 --2- 172.21.15.49:0/3301079540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b10102370 0x7f5b10108c00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 --2- 172.21.15.49:0/3301079540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b101007c0 0x7f5b10100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 -- 172.21.15.49:0/3301079540 >> 172.21.15.49:0/3301079540 conn(0x7f5b100fbd50 msgr2=0x7f5b100fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 -- 172.21.15.49:0/3301079540 shutdown_connections 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 -- 172.21.15.49:0/3301079540 wait complete. 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 Processor -- start 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 -- start start 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b101007c0 0x7f5b10197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b101019b0 0x7f5b10198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b10102370 0x7f5b1019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1010d520 con 0x7f5b10102370 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1010d3a0 con 0x7f5b101007c0 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.880+0000 7f5b154cc640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1010d6a0 con 0x7f5b101019b0 2024-03-26T18:00:02.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b10102370 0x7f5b1019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b101007c0 0x7f5b10197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b10102370 0x7f5b1019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41480/0 (socket says 172.21.15.49:41480) 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b101007c0 0x7f5b10197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43388/0 (socket says 172.21.15.49:43388) 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0f7fe640 1 -- 172.21.15.49:0/314211462 learned_addr learned my addr 172.21.15.49:0/314211462 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0e7fc640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b101019b0 0x7f5b10198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0f7fe640 1 -- 172.21.15.49:0/314211462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b101019b0 msgr2=0x7f5b10198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0f7fe640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b101019b0 0x7f5b10198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0f7fe640 1 -- 172.21.15.49:0/314211462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b101007c0 msgr2=0x7f5b10197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0f7fe640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b101007c0 0x7f5b10197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0f7fe640 1 -- 172.21.15.49:0/314211462 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1019f7e0 con 0x7f5b10102370 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0effd640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b101007c0 0x7f5b10197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0e7fc640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b101019b0 0x7f5b10198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:00:02.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b0f7fe640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b10102370 0x7f5b1019f170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b00036040 tx=0x7f5b000337b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:02.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b07fff640 1 -- 172.21.15.49:0/314211462 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00043020 con 0x7f5b10102370 2024-03-26T18:00:02.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b154cc640 1 -- 172.21.15.49:0/314211462 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1019fa70 con 0x7f5b10102370 2024-03-26T18:00:02.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b07fff640 1 -- 172.21.15.49:0/314211462 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b00003b80 con 0x7f5b10102370 2024-03-26T18:00:02.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b07fff640 1 -- 172.21.15.49:0/314211462 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b000347b0 con 0x7f5b10102370 2024-03-26T18:00:02.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.881+0000 7f5b154cc640 1 -- 172.21.15.49:0/314211462 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b101a0010 con 0x7f5b10102370 2024-03-26T18:00:02.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.882+0000 7f5b154cc640 1 -- 172.21.15.49:0/314211462 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b10107b30 con 0x7f5b10102370 2024-03-26T18:00:02.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.883+0000 7f5b07fff640 1 -- 172.21.15.49:0/314211462 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f5b00046020 con 0x7f5b10102370 2024-03-26T18:00:02.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.884+0000 7f5b07fff640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5afc0a2f20 0x7f5afc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:02.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.884+0000 7f5b0effd640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5afc0a2f20 0x7f5afc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:02.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.884+0000 7f5b07fff640 1 -- 172.21.15.49:0/314211462 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5b000f1dc0 con 0x7f5b10102370 2024-03-26T18:00:02.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.884+0000 7f5b0effd640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5afc0a2f20 0x7f5afc0a53e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b080096c0 tx=0x7f5b08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:02.889+0000 7f5b07fff640 1 -- 172.21.15.49:0/314211462 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5b0004d020 con 0x7f5b10102370 2024-03-26T18:00:03.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.084+0000 7f5b154cc640 1 -- 172.21.15.49:0/314211462 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5b1002d240 con 0x7f5b10102370 2024-03-26T18:00:03.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.085+0000 7f5b07fff640 1 -- 172.21.15.49:0/314211462 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+586 (secure 0 0 0) 0x7f5b000b8e20 con 0x7f5b10102370 2024-03-26T18:00:03.068 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/217046874 2024-03-27T17:52:42.621576+0000 2024-03-26T18:00:03.069 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/442612048 2024-03-27T17:52:42.621576+0000 2024-03-26T18:00:03.069 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1502160114 2024-03-27T17:52:42.621576+0000 2024-03-26T18:00:03.069 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/4249557516 2024-03-27T17:52:42.621576+0000 2024-03-26T18:00:03.069 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2824939577 2024-03-27T17:52:15.448636+0000 2024-03-26T18:00:03.069 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/48778628 2024-03-27T17:52:15.448636+0000 2024-03-26T18:00:03.069 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1915951880 2024-03-27T17:52:15.448636+0000 2024-03-26T18:00:03.069 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/48778628 2024-03-27T17:52:15.448636+0000 2024-03-26T18:00:03.069 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1560035061 2024-03-27T17:52:15.448636+0000 2024-03-26T18:00:03.069 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/4249557516 2024-03-27T17:52:42.621576+0000 2024-03-26T18:00:03.069 INFO:teuthology.orchestra.run.smithi049.stderr:listed 10 entries 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 -- 172.21.15.49:0/314211462 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5afc0a2f20 msgr2=0x7f5afc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5afc0a2f20 0x7f5afc0a53e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b080096c0 tx=0x7f5b08009260 comp rx=0 tx=0).stop 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 -- 172.21.15.49:0/314211462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b10102370 msgr2=0x7f5b1019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b10102370 0x7f5b1019f170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b00036040 tx=0x7f5b000337b0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 -- 172.21.15.49:0/314211462 shutdown_connections 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5afc0a2f20 0x7f5afc0a53e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b101019b0 0x7f5b10198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b10102370 0x7f5b1019f170 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 --2- 172.21.15.49:0/314211462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b101007c0 0x7f5b10197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 -- 172.21.15.49:0/314211462 >> 172.21.15.49:0/314211462 conn(0x7f5b100fbd50 msgr2=0x7f5b10104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 -- 172.21.15.49:0/314211462 shutdown_connections 2024-03-26T18:00:03.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.087+0000 7f5b154cc640 1 -- 172.21.15.49:0/314211462 wait complete. 2024-03-26T18:00:03.081 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2024-03-26T18:00:03.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:00:03.082 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-26T18:00:03.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi049.stderr:ceph-fuse[75942]: fuse finished with error 0 and tester_r 0 2024-03-26T18:00:03.113 INFO:teuthology.orchestra.run:waiting for 300 2024-03-26T18:00:03.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T18:00:03.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:00:03.189 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-26T18:00:03.214 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:00:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.379+0000 7fbb90b0c640 1 -- 172.21.15.49:0/791837621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb8c1018f0 msgr2=0x7fbb8c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.379+0000 7fbb90b0c640 1 --2- 172.21.15.49:0/791837621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb8c1018f0 0x7fbb8c101d10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009ca0 tx=0x7fbb7c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 -- 172.21.15.49:0/791837621 shutdown_connections 2024-03-26T18:00:03.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 --2- 172.21.15.49:0/791837621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb8c102ae0 0x7fbb8c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 --2- 172.21.15.49:0/791837621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb8c1034a0 0x7fbb8c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 --2- 172.21.15.49:0/791837621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb8c1018f0 0x7fbb8c101d10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 -- 172.21.15.49:0/791837621 >> 172.21.15.49:0/791837621 conn(0x7fbb8c0fcea0 msgr2=0x7fbb8c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 -- 172.21.15.49:0/791837621 shutdown_connections 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 -- 172.21.15.49:0/791837621 wait complete. 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 Processor -- start 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 -- start start 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb8c1018f0 0x7fbb8c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb8c102ae0 0x7fbb8c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb8c1034a0 0x7fbb8c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c10d540 con 0x7fbb8c1034a0 2024-03-26T18:00:03.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c10d3c0 con 0x7fbb8c102ae0 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.380+0000 7fbb90b0c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c10d6c0 con 0x7fbb8c1018f0 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb8ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb8c1034a0 0x7fbb8c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb8ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb8c1034a0 0x7fbb8c19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41492/0 (socket says 172.21.15.49:41492) 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb8ad9d640 1 -- 172.21.15.49:0/2498337603 learned_addr learned my addr 172.21.15.49:0/2498337603 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb89d9b640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb8c102ae0 0x7fbb8c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb8a59c640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb8c1018f0 0x7fbb8c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb8ad9d640 1 -- 172.21.15.49:0/2498337603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb8c1018f0 msgr2=0x7fbb8c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb8ad9d640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb8c1018f0 0x7fbb8c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb8ad9d640 1 -- 172.21.15.49:0/2498337603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb8c102ae0 msgr2=0x7fbb8c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb8ad9d640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb8c102ae0 0x7fbb8c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb8ad9d640 1 -- 172.21.15.49:0/2498337603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8c19f740 con 0x7fbb8c1034a0 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb8ad9d640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb8c1034a0 0x7fbb8c19f0d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e970 tx=0x7fbb8400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb837fe640 1 -- 172.21.15.49:0/2498337603 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400ccc0 con 0x7fbb8c1034a0 2024-03-26T18:00:03.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb837fe640 1 -- 172.21.15.49:0/2498337603 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb8400ce60 con 0x7fbb8c1034a0 2024-03-26T18:00:03.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb90b0c640 1 -- 172.21.15.49:0/2498337603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8c19fa30 con 0x7fbb8c1034a0 2024-03-26T18:00:03.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb837fe640 1 -- 172.21.15.49:0/2498337603 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84010430 con 0x7fbb8c1034a0 2024-03-26T18:00:03.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.381+0000 7fbb90b0c640 1 -- 172.21.15.49:0/2498337603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8c19ff30 con 0x7fbb8c1034a0 2024-03-26T18:00:03.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.382+0000 7fbb90b0c640 1 -- 172.21.15.49:0/2498337603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8c108bb0 con 0x7fbb8c1034a0 2024-03-26T18:00:03.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.383+0000 7fbb837fe640 1 -- 172.21.15.49:0/2498337603 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fbb84005000 con 0x7fbb8c1034a0 2024-03-26T18:00:03.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.384+0000 7fbb837fe640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbb780a2f20 0x7fbb780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:03.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.384+0000 7fbb8a59c640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbb780a2f20 0x7fbb780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:03.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.384+0000 7fbb837fe640 1 -- 172.21.15.49:0/2498337603 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbb840cfc20 con 0x7fbb8c1034a0 2024-03-26T18:00:03.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.385+0000 7fbb8a59c640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbb780a2f20 0x7fbb780a53e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c037040 tx=0x7fbb7c0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:03.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.388+0000 7fbb837fe640 1 -- 172.21.15.49:0/2498337603 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbb84053740 con 0x7fbb8c1034a0 2024-03-26T18:00:03.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.573+0000 7fbb90b0c640 1 -- 172.21.15.49:0/2498337603 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb8c0624c0 con 0x7fbb8c1034a0 2024-03-26T18:00:03.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.574+0000 7fbb837fe640 1 -- 172.21.15.49:0/2498337603 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24136 (secure 0 0 0) 0x7fbb8409c810 con 0x7fbb8c1034a0 2024-03-26T18:00:03.558 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:00:03.558 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:51.836345+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:00:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 -- 172.21.15.49:0/2498337603 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbb780a2f20 msgr2=0x7fbb780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbb780a2f20 0x7fbb780a53e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c037040 tx=0x7fbb7c0093f0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 -- 172.21.15.49:0/2498337603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb8c1034a0 msgr2=0x7fbb8c19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb8c1034a0 0x7fbb8c19f0d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e970 tx=0x7fbb8400ed70 comp rx=0 tx=0).stop 2024-03-26T18:00:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 -- 172.21.15.49:0/2498337603 shutdown_connections 2024-03-26T18:00:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbb780a2f20 0x7fbb780a53e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb8c1034a0 0x7fbb8c19f0d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb8c102ae0 0x7fbb8c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 --2- 172.21.15.49:0/2498337603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb8c1018f0 0x7fbb8c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 -- 172.21.15.49:0/2498337603 >> 172.21.15.49:0/2498337603 conn(0x7fbb8c0fcea0 msgr2=0x7fbb8c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 -- 172.21.15.49:0/2498337603 shutdown_connections 2024-03-26T18:00:03.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.577+0000 7fbb90b0c640 1 -- 172.21.15.49:0/2498337603 wait complete. 2024-03-26T18:00:03.572 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:00:03.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.696+0000 7f618ffc6640 1 -- 172.21.15.49:0/2669280619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6188100b30 msgr2=0x7f6188100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.696+0000 7f618ffc6640 1 --2- 172.21.15.49:0/2669280619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6188100b30 0x7f6188100f90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6178009900 tx=0x7f617802bcd0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 -- 172.21.15.49:0/2669280619 shutdown_connections 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 --2- 172.21.15.49:0/2669280619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61881014d0 0x7f6188101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 --2- 172.21.15.49:0/2669280619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6188100b30 0x7f6188100f90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 --2- 172.21.15.49:0/2669280619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61881001d0 0x7f61881005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 -- 172.21.15.49:0/2669280619 >> 172.21.15.49:0/2669280619 conn(0x7f61880fbd50 msgr2=0x7f61880fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 -- 172.21.15.49:0/2669280619 shutdown_connections 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 -- 172.21.15.49:0/2669280619 wait complete. 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 Processor -- start 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 -- start start 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61881001d0 0x7f6188199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6188100b30 0x7f618819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61881014d0 0x7f61881a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:03.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f618810f760 con 0x7f6188100b30 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f618810f5e0 con 0x7f61881001d0 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.697+0000 7f618ffc6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f618810f8e0 con 0x7f61881014d0 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618d561640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6188100b30 0x7f618819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618d561640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6188100b30 0x7f618819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41520/0 (socket says 172.21.15.49:41520) 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618dd62640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61881001d0 0x7f6188199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618d561640 1 -- 172.21.15.49:0/1748852493 learned_addr learned my addr 172.21.15.49:0/1748852493 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618e563640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61881014d0 0x7f61881a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618d561640 1 -- 172.21.15.49:0/1748852493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61881014d0 msgr2=0x7f61881a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618d561640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61881014d0 0x7f61881a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618d561640 1 -- 172.21.15.49:0/1748852493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61881001d0 msgr2=0x7f6188199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618d561640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61881001d0 0x7f6188199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618d561640 1 -- 172.21.15.49:0/1748852493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61881a1a80 con 0x7f6188100b30 2024-03-26T18:00:03.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618dd62640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61881001d0 0x7f6188199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618d561640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6188100b30 0x7f618819a3a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f617802efd0 tx=0x7f6178005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f617effd640 1 -- 172.21.15.49:0/1748852493 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178041070 con 0x7f6188100b30 2024-03-26T18:00:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f617effd640 1 -- 172.21.15.49:0/1748852493 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61780044d0 con 0x7f6188100b30 2024-03-26T18:00:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618ffc6640 1 -- 172.21.15.49:0/1748852493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61881a1d10 con 0x7f6188100b30 2024-03-26T18:00:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f617effd640 1 -- 172.21.15.49:0/1748852493 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178034440 con 0x7f6188100b30 2024-03-26T18:00:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.698+0000 7f618ffc6640 1 -- 172.21.15.49:0/1748852493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61881a21f0 con 0x7f6188100b30 2024-03-26T18:00:03.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.699+0000 7f618ffc6640 1 -- 172.21.15.49:0/1748852493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618810b0f0 con 0x7f6188100b30 2024-03-26T18:00:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.700+0000 7f617effd640 1 -- 172.21.15.49:0/1748852493 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f6178044070 con 0x7f6188100b30 2024-03-26T18:00:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.701+0000 7f617effd640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61740a2fa0 0x7f61740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.701+0000 7f617effd640 1 -- 172.21.15.49:0/1748852493 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f61780f1080 con 0x7f6188100b30 2024-03-26T18:00:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.701+0000 7f618dd62640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61740a2fa0 0x7f61740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:03.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.701+0000 7f618dd62640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61740a2fa0 0x7f61740a5460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6184005fd0 tx=0x7f618400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:03.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.705+0000 7f617effd640 1 -- 172.21.15.49:0/1748852493 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f61780bdc70 con 0x7f6188100b30 2024-03-26T18:00:03.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:03 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/50420341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:00:03.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:03 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/314211462' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:00:03.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:03 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/50420341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:00:03.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:03 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/314211462' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:00:03.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:03 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/50420341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:00:03.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:03 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/314211462' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:00:03.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.890+0000 7f618ffc6640 1 -- 172.21.15.49:0/1748852493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61880624f0 con 0x7f6188100b30 2024-03-26T18:00:03.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.892+0000 7f617effd640 1 -- 172.21.15.49:0/1748852493 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24136 (secure 0 0 0) 0x7f61780bd570 con 0x7f6188100b30 2024-03-26T18:00:03.876 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:00:03.876 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:51.836345+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:00:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 -- 172.21.15.49:0/1748852493 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61740a2fa0 msgr2=0x7f61740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61740a2fa0 0x7f61740a5460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6184005fd0 tx=0x7f618400b390 comp rx=0 tx=0).stop 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 -- 172.21.15.49:0/1748852493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6188100b30 msgr2=0x7f618819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6188100b30 0x7f618819a3a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f617802efd0 tx=0x7f6178005d50 comp rx=0 tx=0).stop 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 -- 172.21.15.49:0/1748852493 shutdown_connections 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f61740a2fa0 0x7f61740a5460 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61881014d0 0x7f61881a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6188100b30 0x7f618819a3a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 --2- 172.21.15.49:0/1748852493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61881001d0 0x7f6188199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 -- 172.21.15.49:0/1748852493 >> 172.21.15.49:0/1748852493 conn(0x7f61880fbd50 msgr2=0x7f61880fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 -- 172.21.15.49:0/1748852493 shutdown_connections 2024-03-26T18:00:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:03.894+0000 7f618ffc6640 1 -- 172.21.15.49:0/1748852493 wait complete. 2024-03-26T18:00:03.889 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-26T18:00:04.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.017+0000 7f193459e640 1 -- 172.21.15.49:0/4269660653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c1007e0 msgr2=0x7f192c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:04.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.017+0000 7f193459e640 1 --2- 172.21.15.49:0/4269660653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c1007e0 0x7f192c100c00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1928009900 tx=0x7f192802bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:04.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.017+0000 7f193459e640 1 -- 172.21.15.49:0/4269660653 shutdown_connections 2024-03-26T18:00:04.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.017+0000 7f193459e640 1 --2- 172.21.15.49:0/4269660653 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f192c102390 0x7f192c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.017+0000 7f193459e640 1 --2- 172.21.15.49:0/4269660653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f192c1019d0 0x7f192c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.017+0000 7f193459e640 1 --2- 172.21.15.49:0/4269660653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c1007e0 0x7f192c100c00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.017+0000 7f193459e640 1 -- 172.21.15.49:0/4269660653 >> 172.21.15.49:0/4269660653 conn(0x7f192c0fbd70 msgr2=0x7f192c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.017+0000 7f193459e640 1 -- 172.21.15.49:0/4269660653 shutdown_connections 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.017+0000 7f193459e640 1 -- 172.21.15.49:0/4269660653 wait complete. 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f193459e640 1 Processor -- start 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f193459e640 1 -- start start 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f193459e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f192c1007e0 0x7f192c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f193459e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f192c1019d0 0x7f192c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f193459e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c102390 0x7f192c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f193459e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f192c10d540 con 0x7f192c102390 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f193459e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f192c10d3c0 con 0x7f192c1007e0 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f193459e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f192c10d6c0 con 0x7f192c1019d0 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f1932b3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c102390 0x7f192c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f1932b3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c102390 0x7f192c19f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41538/0 (socket says 172.21.15.49:41538) 2024-03-26T18:00:04.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f1932b3b640 1 -- 172.21.15.49:0/84699182 learned_addr learned my addr 172.21.15.49:0/84699182 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f193233a640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f192c1007e0 0x7f192c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f1931b39640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f192c1019d0 0x7f192c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f1932b3b640 1 -- 172.21.15.49:0/84699182 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f192c1019d0 msgr2=0x7f192c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f1932b3b640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f192c1019d0 0x7f192c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f1932b3b640 1 -- 172.21.15.49:0/84699182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f192c1007e0 msgr2=0x7f192c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f1932b3b640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f192c1007e0 0x7f192c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f1932b3b640 1 -- 172.21.15.49:0/84699182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192c19f7b0 con 0x7f192c102390 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.018+0000 7f193233a640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f192c1007e0 0x7f192c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.019+0000 7f1932b3b640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c102390 0x7f192c19f140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f192400cce0 tx=0x7f192400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.019+0000 7f19237fe640 1 -- 172.21.15.49:0/84699182 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192400f450 con 0x7f192c102390 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.019+0000 7f19237fe640 1 -- 172.21.15.49:0/84699182 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f192400fad0 con 0x7f192c102390 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.019+0000 7f193459e640 1 -- 172.21.15.49:0/84699182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192c19faa0 con 0x7f192c102390 2024-03-26T18:00:04.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.019+0000 7f19237fe640 1 -- 172.21.15.49:0/84699182 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924018c10 con 0x7f192c102390 2024-03-26T18:00:04.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.019+0000 7f193459e640 1 -- 172.21.15.49:0/84699182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f192c19ffa0 con 0x7f192c102390 2024-03-26T18:00:04.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.020+0000 7f193459e640 1 -- 172.21.15.49:0/84699182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f192c107b80 con 0x7f192c102390 2024-03-26T18:00:04.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.021+0000 7f19237fe640 1 -- 172.21.15.49:0/84699182 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f192400f5f0 con 0x7f192c102390 2024-03-26T18:00:04.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.022+0000 7f19237fe640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f19180a2f20 0x7f19180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:04.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.022+0000 7f193233a640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f19180a2f20 0x7f19180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:04.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.022+0000 7f19237fe640 1 -- 172.21.15.49:0/84699182 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f19240cfdb0 con 0x7f192c102390 2024-03-26T18:00:04.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.022+0000 7f193233a640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f19180a2f20 0x7f19180a53e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f1928005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:04.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.025+0000 7f19237fe640 1 -- 172.21.15.49:0/84699182 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1924014030 con 0x7f192c102390 2024-03-26T18:00:04.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.248+0000 7f193459e640 1 -- 172.21.15.49:0/84699182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f192c0624c0 con 0x7f192c102390 2024-03-26T18:00:04.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.490+0000 7f19237fe640 1 -- 172.21.15.49:0/84699182 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f192409ca30 con 0x7f192c102390 2024-03-26T18:00:04.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 -- 172.21.15.49:0/84699182 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f19180a2f20 msgr2=0x7f19180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:04.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f19180a2f20 0x7f19180a53e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f1928005fd0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 -- 172.21.15.49:0/84699182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c102390 msgr2=0x7f192c19f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:04.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c102390 0x7f192c19f140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f192400cce0 tx=0x7f192400a620 comp rx=0 tx=0).stop 2024-03-26T18:00:04.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 -- 172.21.15.49:0/84699182 shutdown_connections 2024-03-26T18:00:04.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f19180a2f20 0x7f19180a53e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f192c1019d0 0x7f192c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f192c102390 0x7f192c19f140 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 --2- 172.21.15.49:0/84699182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f192c1007e0 0x7f192c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 -- 172.21.15.49:0/84699182 >> 172.21.15.49:0/84699182 conn(0x7f192c0fbd70 msgr2=0x7f192c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:04.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 -- 172.21.15.49:0/84699182 shutdown_connections 2024-03-26T18:00:04.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.493+0000 7f193459e640 1 -- 172.21.15.49:0/84699182 wait complete. 2024-03-26T18:00:04.488 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-26T18:00:04.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-26T18:00:04.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.617+0000 7fc630ea1640 1 -- 172.21.15.49:0/2845820678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc62c107600 msgr2=0x7fc62c1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:04.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.617+0000 7fc630ea1640 1 --2- 172.21.15.49:0/2845820678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc62c107600 0x7fc62c1099f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc624009ca0 tx=0x7fc62402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.617+0000 7fc630ea1640 1 -- 172.21.15.49:0/2845820678 shutdown_connections 2024-03-26T18:00:04.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.617+0000 7fc630ea1640 1 --2- 172.21.15.49:0/2845820678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c104cd0 0x7fc62c1070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.617+0000 7fc630ea1640 1 --2- 172.21.15.49:0/2845820678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc62c107600 0x7fc62c1099f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.617+0000 7fc630ea1640 1 --2- 172.21.15.49:0/2845820678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc62c1023a0 0x7fc62c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.617+0000 7fc630ea1640 1 -- 172.21.15.49:0/2845820678 >> 172.21.15.49:0/2845820678 conn(0x7fc62c0fbd50 msgr2=0x7fc62c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.617+0000 7fc630ea1640 1 -- 172.21.15.49:0/2845820678 shutdown_connections 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.617+0000 7fc630ea1640 1 -- 172.21.15.49:0/2845820678 wait complete. 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc630ea1640 1 Processor -- start 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc630ea1640 1 -- start start 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc630ea1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc62c1023a0 0x7fc62c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc630ea1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c104cd0 0x7fc62c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc630ea1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc62c107600 0x7fc62c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:04.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc630ea1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10d560 con 0x7fc62c107600 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc630ea1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10d3e0 con 0x7fc62c1023a0 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc630ea1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10d6e0 con 0x7fc62c104cd0 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc62ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc62c107600 0x7fc62c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc62ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc62c107600 0x7fc62c19f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41548/0 (socket says 172.21.15.49:41548) 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc62a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc62c1023a0 0x7fc62c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc62ad9d640 1 -- 172.21.15.49:0/2239099468 learned_addr learned my addr 172.21.15.49:0/2239099468 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.618+0000 7fc623fff640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c104cd0 0x7fc62c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc62ad9d640 1 -- 172.21.15.49:0/2239099468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c104cd0 msgr2=0x7fc62c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc62ad9d640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c104cd0 0x7fc62c198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc62ad9d640 1 -- 172.21.15.49:0/2239099468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc62c1023a0 msgr2=0x7fc62c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc62ad9d640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc62c1023a0 0x7fc62c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc62ad9d640 1 -- 172.21.15.49:0/2239099468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c19f800 con 0x7fc62c107600 2024-03-26T18:00:04.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc62a59c640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc62c1023a0 0x7fc62c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc62ad9d640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc62c107600 0x7fc62c19f190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc624037040 tx=0x7fc624032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc6237fe640 1 -- 172.21.15.49:0/2239099468 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624041020 con 0x7fc62c107600 2024-03-26T18:00:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc630ea1640 1 -- 172.21.15.49:0/2239099468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c19fa90 con 0x7fc62c107600 2024-03-26T18:00:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc6237fe640 1 -- 172.21.15.49:0/2239099468 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624034d90 con 0x7fc62c107600 2024-03-26T18:00:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc6237fe640 1 -- 172.21.15.49:0/2239099468 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624039040 con 0x7fc62c107600 2024-03-26T18:00:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.619+0000 7fc630ea1640 1 -- 172.21.15.49:0/2239099468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c19ff80 con 0x7fc62c107600 2024-03-26T18:00:04.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.620+0000 7fc630ea1640 1 -- 172.21.15.49:0/2239099468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62c0fd840 con 0x7fc62c107600 2024-03-26T18:00:04.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.621+0000 7fc6237fe640 1 -- 172.21.15.49:0/2239099468 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fc624044020 con 0x7fc62c107600 2024-03-26T18:00:04.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.622+0000 7fc6237fe640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6180a2f20 0x7fc6180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:04.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.622+0000 7fc62a59c640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6180a2f20 0x7fc6180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:04.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.622+0000 7fc6237fe640 1 -- 172.21.15.49:0/2239099468 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc6240f15c0 con 0x7fc62c107600 2024-03-26T18:00:04.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.622+0000 7fc62a59c640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6180a2f20 0x7fc6180a53e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009fc0 tx=0x7fc61c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:04.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.626+0000 7fc6237fe640 1 -- 172.21.15.49:0/2239099468 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc62404b020 con 0x7fc62c107600 2024-03-26T18:00:04.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:04 smithi115 ceph-mon[36174]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 299 B/s wr, 0 op/s 2024-03-26T18:00:04.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:04 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2498337603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:04.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:04 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1748852493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:04.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:04 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/84699182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T18:00:04.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:04 smithi049 ceph-mon[32331]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 299 B/s wr, 0 op/s 2024-03-26T18:00:04.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:04 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2498337603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:04.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:04 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1748852493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:04.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:04 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/84699182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T18:00:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:04.853+0000 7fc630ea1640 1 -- 172.21.15.49:0/2239099468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fc62c0624c0 con 0x7fc62c107600 2024-03-26T18:00:04.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:04 smithi171 ceph-mon[36033]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 299 B/s wr, 0 op/s 2024-03-26T18:00:04.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:04 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2498337603' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:04.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:04 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1748852493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:04.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:04 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/84699182' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T18:00:05.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.495+0000 7fc6237fe640 1 -- 172.21.15.49:0/2239099468 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fc6240be1b0 con 0x7fc62c107600 2024-03-26T18:00:05.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.498+0000 7fc630ea1640 1 -- 172.21.15.49:0/2239099468 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6180a2f20 msgr2=0x7fc6180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:05.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6180a2f20 0x7fc6180a53e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009fc0 tx=0x7fc61c005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:05.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 -- 172.21.15.49:0/2239099468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc62c107600 msgr2=0x7fc62c19f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:05.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc62c107600 0x7fc62c19f190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc624037040 tx=0x7fc624032f20 comp rx=0 tx=0).stop 2024-03-26T18:00:05.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 -- 172.21.15.49:0/2239099468 shutdown_connections 2024-03-26T18:00:05.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc6180a2f20 0x7fc6180a53e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:05.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc62c104cd0 0x7fc62c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:05.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc62c107600 0x7fc62c19f190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:05.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 --2- 172.21.15.49:0/2239099468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc62c1023a0 0x7fc62c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:05.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 -- 172.21.15.49:0/2239099468 >> 172.21.15.49:0/2239099468 conn(0x7fc62c0fbd50 msgr2=0x7fc62c102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:05.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 -- 172.21.15.49:0/2239099468 shutdown_connections 2024-03-26T18:00:05.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.499+0000 7fc630ea1640 1 -- 172.21.15.49:0/2239099468 wait complete. 2024-03-26T18:00:05.501 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-26T18:00:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.634+0000 7f3a14859640 1 -- 172.21.15.49:0/2724333150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a10102390 msgr2=0x7f3a10108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.634+0000 7f3a14859640 1 --2- 172.21.15.49:0/2724333150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a10102390 0x7f3a10108c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009ca0 tx=0x7f3a0802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.634+0000 7f3a14859640 1 -- 172.21.15.49:0/2724333150 shutdown_connections 2024-03-26T18:00:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.634+0000 7f3a14859640 1 --2- 172.21.15.49:0/2724333150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a101019d0 0x7f3a10101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.634+0000 7f3a14859640 1 --2- 172.21.15.49:0/2724333150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a10102390 0x7f3a10108c20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.634+0000 7f3a14859640 1 --2- 172.21.15.49:0/2724333150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a101007e0 0x7f3a10100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:05.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.634+0000 7f3a14859640 1 -- 172.21.15.49:0/2724333150 >> 172.21.15.49:0/2724333150 conn(0x7f3a100fbd70 msgr2=0x7f3a100fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:05.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.634+0000 7f3a14859640 1 -- 172.21.15.49:0/2724333150 shutdown_connections 2024-03-26T18:00:05.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.634+0000 7f3a14859640 1 -- 172.21.15.49:0/2724333150 wait complete. 2024-03-26T18:00:05.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a14859640 1 Processor -- start 2024-03-26T18:00:05.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a14859640 1 -- start start 2024-03-26T18:00:05.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a14859640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a101007e0 0x7f3a1006aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a14859640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a101019d0 0x7f3a1019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a14859640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a10102390 0x7f3a1019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a14859640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1010c4c0 con 0x7f3a101007e0 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a14859640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1010c340 con 0x7f3a10102390 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a14859640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1010c640 con 0x7f3a101019d0 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a0e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a101007e0 0x7f3a1006aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a0e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a101007e0 0x7f3a1006aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41568/0 (socket says 172.21.15.49:41568) 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a0e59c640 1 -- 172.21.15.49:0/2182501733 learned_addr learned my addr 172.21.15.49:0/2182501733 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a0ed9d640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a10102390 0x7f3a1019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.635+0000 7f3a05d9b640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a101019d0 0x7f3a1019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:05.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a0e59c640 1 -- 172.21.15.49:0/2182501733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a101019d0 msgr2=0x7f3a1019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a0e59c640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a101019d0 0x7f3a1019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a0e59c640 1 -- 172.21.15.49:0/2182501733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a10102390 msgr2=0x7f3a1019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a0e59c640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a10102390 0x7f3a1019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a0e59c640 1 -- 172.21.15.49:0/2182501733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1019f810 con 0x7f3a101007e0 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a0ed9d640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a10102390 0x7f3a1019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a0e59c640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a101007e0 0x7f3a1006aa20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000e9c0 tx=0x7f3a0000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a07fff640 1 -- 172.21.15.49:0/2182501733 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0000cd80 con 0x7f3a101007e0 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a07fff640 1 -- 172.21.15.49:0/2182501733 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a000035f0 con 0x7f3a101007e0 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a14859640 1 -- 172.21.15.49:0/2182501733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1019fb00 con 0x7f3a101007e0 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a07fff640 1 -- 172.21.15.49:0/2182501733 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00010690 con 0x7f3a101007e0 2024-03-26T18:00:05.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.636+0000 7f3a14859640 1 -- 172.21.15.49:0/2182501733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a101a0050 con 0x7f3a101007e0 2024-03-26T18:00:05.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.637+0000 7f3a14859640 1 -- 172.21.15.49:0/2182501733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a10107aa0 con 0x7f3a101007e0 2024-03-26T18:00:05.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.638+0000 7f3a07fff640 1 -- 172.21.15.49:0/2182501733 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f3a000037a0 con 0x7f3a101007e0 2024-03-26T18:00:05.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.639+0000 7f3a07fff640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39fc0a2f20 0x7f39fc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:05.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.639+0000 7f3a05d9b640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39fc0a2f20 0x7f39fc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:05.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.639+0000 7f3a07fff640 1 -- 172.21.15.49:0/2182501733 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3a000cfab0 con 0x7f3a101007e0 2024-03-26T18:00:05.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.639+0000 7f3a05d9b640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39fc0a2f20 0x7f39fc0a53e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a1006a790 tx=0x7f39f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:05.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.643+0000 7f3a07fff640 1 -- 172.21.15.49:0/2182501733 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3a0009c6a0 con 0x7f3a101007e0 2024-03-26T18:00:05.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:05 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/84699182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T18:00:05.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:05 smithi115 ceph-mon[36174]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:00:05.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:05 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2239099468' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T18:00:05.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:05 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/84699182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T18:00:05.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:05 smithi049 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:00:05.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:05 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2239099468' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T18:00:05.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:05.866+0000 7f3a14859640 1 -- 172.21.15.49:0/2182501733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f3a1002d240 con 0x7f3a101007e0 2024-03-26T18:00:05.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:05 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/84699182' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T18:00:05.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:05 smithi171 ceph-mon[36033]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:00:05.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:05 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2239099468' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T18:00:06.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.503+0000 7f3a07fff640 1 -- 172.21.15.49:0/2182501733 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f3a0009bfa0 con 0x7f3a101007e0 2024-03-26T18:00:06.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.505+0000 7f3a14859640 1 -- 172.21.15.49:0/2182501733 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39fc0a2f20 msgr2=0x7f39fc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.505+0000 7f3a14859640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39fc0a2f20 0x7f39fc0a53e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a1006a790 tx=0x7f39f8008040 comp rx=0 tx=0).stop 2024-03-26T18:00:06.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.505+0000 7f3a14859640 1 -- 172.21.15.49:0/2182501733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a101007e0 msgr2=0x7f3a1006aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.505+0000 7f3a14859640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a101007e0 0x7f3a1006aa20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000e9c0 tx=0x7f3a0000ee80 comp rx=0 tx=0).stop 2024-03-26T18:00:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.506+0000 7f3a14859640 1 -- 172.21.15.49:0/2182501733 shutdown_connections 2024-03-26T18:00:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.506+0000 7f3a14859640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39fc0a2f20 0x7f39fc0a53e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.506+0000 7f3a14859640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a101019d0 0x7f3a1019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.506+0000 7f3a14859640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a10102390 0x7f3a1019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.506+0000 7f3a14859640 1 --2- 172.21.15.49:0/2182501733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a101007e0 0x7f3a1006aa20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.506+0000 7f3a14859640 1 -- 172.21.15.49:0/2182501733 >> 172.21.15.49:0/2182501733 conn(0x7f3a100fbd70 msgr2=0x7f3a10104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.506+0000 7f3a14859640 1 -- 172.21.15.49:0/2182501733 shutdown_connections 2024-03-26T18:00:06.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.506+0000 7f3a14859640 1 -- 172.21.15.49:0/2182501733 wait complete. 2024-03-26T18:00:06.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-26T18:00:06.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.641+0000 7f6fcc84b640 1 -- 172.21.15.49:0/2223924131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fc8102370 msgr2=0x7f6fc8108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.641+0000 7f6fcc84b640 1 --2- 172.21.15.49:0/2223924131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fc8102370 0x7f6fc8108c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009ca0 tx=0x7f6fc002c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.642+0000 7f6fcc84b640 1 -- 172.21.15.49:0/2223924131 shutdown_connections 2024-03-26T18:00:06.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.642+0000 7f6fcc84b640 1 --2- 172.21.15.49:0/2223924131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fc81019b0 0x7f6fc8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.642+0000 7f6fcc84b640 1 --2- 172.21.15.49:0/2223924131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fc8102370 0x7f6fc8108c00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.642+0000 7f6fcc84b640 1 --2- 172.21.15.49:0/2223924131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fc81007c0 0x7f6fc8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.642+0000 7f6fcc84b640 1 -- 172.21.15.49:0/2223924131 >> 172.21.15.49:0/2223924131 conn(0x7f6fc80fbd50 msgr2=0x7f6fc80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.642+0000 7f6fcc84b640 1 -- 172.21.15.49:0/2223924131 shutdown_connections 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.642+0000 7f6fcc84b640 1 -- 172.21.15.49:0/2223924131 wait complete. 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.642+0000 7f6fcc84b640 1 Processor -- start 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.642+0000 7f6fcc84b640 1 -- start start 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fcc84b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fc81007c0 0x7f6fc8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fcc84b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fc81019b0 0x7f6fc8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fcc84b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fc8102370 0x7f6fc819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fcc84b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc810d520 con 0x7f6fc8102370 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fcc84b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc810d3a0 con 0x7f6fc81007c0 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fcc84b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc810d6a0 con 0x7f6fc81019b0 2024-03-26T18:00:06.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fc8102370 0x7f6fc819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fc8102370 0x7f6fc819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41582/0 (socket says 172.21.15.49:41582) 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc6d9d640 1 -- 172.21.15.49:0/1627273914 learned_addr learned my addr 172.21.15.49:0/1627273914 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc659c640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fc81007c0 0x7f6fc8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc5d9b640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fc81019b0 0x7f6fc8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc6d9d640 1 -- 172.21.15.49:0/1627273914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fc81019b0 msgr2=0x7f6fc8198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc6d9d640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fc81019b0 0x7f6fc8198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc6d9d640 1 -- 172.21.15.49:0/1627273914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fc81007c0 msgr2=0x7f6fc8197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc6d9d640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fc81007c0 0x7f6fc8197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc6d9d640 1 -- 172.21.15.49:0/1627273914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc819f720 con 0x7f6fc8102370 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc659c640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fc81007c0 0x7f6fc8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fc6d9d640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fc8102370 0x7f6fc819f0b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0037040 tx=0x7f6fc0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fb77fe640 1 -- 172.21.15.49:0/1627273914 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0041020 con 0x7f6fc8102370 2024-03-26T18:00:06.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fb77fe640 1 -- 172.21.15.49:0/1627273914 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fc0034cd0 con 0x7f6fc8102370 2024-03-26T18:00:06.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fcc84b640 1 -- 172.21.15.49:0/1627273914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc819f9b0 con 0x7f6fc8102370 2024-03-26T18:00:06.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fb77fe640 1 -- 172.21.15.49:0/1627273914 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0039040 con 0x7f6fc8102370 2024-03-26T18:00:06.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.643+0000 7f6fcc84b640 1 -- 172.21.15.49:0/1627273914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc819fea0 con 0x7f6fc8102370 2024-03-26T18:00:06.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.645+0000 7f6fcc84b640 1 -- 172.21.15.49:0/1627273914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc8107a80 con 0x7f6fc8102370 2024-03-26T18:00:06.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.646+0000 7f6fb77fe640 1 -- 172.21.15.49:0/1627273914 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f6fc0044020 con 0x7f6fc8102370 2024-03-26T18:00:06.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.646+0000 7f6fb77fe640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6fb00a2f20 0x7f6fb00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.646+0000 7f6fc659c640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6fb00a2f20 0x7f6fb00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.646+0000 7f6fb77fe640 1 -- 172.21.15.49:0/1627273914 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6fc00f15a0 con 0x7f6fc8102370 2024-03-26T18:00:06.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.647+0000 7f6fc659c640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6fb00a2f20 0x7f6fb00a53e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8101810 tx=0x7f6fb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:06.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.651+0000 7f6fb77fe640 1 -- 172.21.15.49:0/1627273914 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6fc004b020 con 0x7f6fc8102370 2024-03-26T18:00:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:06 smithi115 ceph-mon[36174]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-26T18:00:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:06 smithi115 ceph-mon[36174]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T18:00:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:06 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2239099468' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T18:00:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:06 smithi115 ceph-mon[36174]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T18:00:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:06 smithi115 ceph-mon[36174]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:00:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:06 smithi115 ceph-mon[36174]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-26T18:00:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:06 smithi115 ceph-mon[36174]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-26T18:00:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:06 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2182501733' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T18:00:06.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:06 smithi049 ceph-mon[32331]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-26T18:00:06.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:06 smithi049 ceph-mon[32331]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T18:00:06.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:06 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2239099468' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T18:00:06.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:06 smithi049 ceph-mon[32331]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T18:00:06.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:06 smithi049 ceph-mon[32331]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:00:06.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:06 smithi049 ceph-mon[32331]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-26T18:00:06.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:06 smithi049 ceph-mon[32331]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-26T18:00:06.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:06 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2182501733' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T18:00:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:06 smithi171 ceph-mon[36033]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-26T18:00:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:06 smithi171 ceph-mon[36033]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T18:00:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:06 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2239099468' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T18:00:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:06 smithi171 ceph-mon[36033]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T18:00:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:06 smithi171 ceph-mon[36033]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T18:00:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:06 smithi171 ceph-mon[36033]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-26T18:00:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:06 smithi171 ceph-mon[36033]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-26T18:00:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:06 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2182501733' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T18:00:06.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:06.878+0000 7f6fcc84b640 1 -- 172.21.15.49:0/1627273914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f6fc802d420 con 0x7f6fc8102370 2024-03-26T18:00:07.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.433+0000 7f6fb77fe640 1 -- 172.21.15.49:0/1627273914 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f6fc00be190 con 0x7f6fc8102370 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 -- 172.21.15.49:0/1627273914 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6fb00a2f20 msgr2=0x7f6fb00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6fb00a2f20 0x7f6fb00a53e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8101810 tx=0x7f6fb8005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 -- 172.21.15.49:0/1627273914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fc8102370 msgr2=0x7f6fc819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fc8102370 0x7f6fc819f0b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0037040 tx=0x7f6fc0032f20 comp rx=0 tx=0).stop 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 -- 172.21.15.49:0/1627273914 shutdown_connections 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6fb00a2f20 0x7f6fb00a53e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6fc81019b0 0x7f6fc8198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fc8102370 0x7f6fc819f0b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 --2- 172.21.15.49:0/1627273914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fc81007c0 0x7f6fc8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 -- 172.21.15.49:0/1627273914 >> 172.21.15.49:0/1627273914 conn(0x7f6fc80fbd50 msgr2=0x7f6fc8104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:07.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 -- 172.21.15.49:0/1627273914 shutdown_connections 2024-03-26T18:00:07.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.435+0000 7f6fcc84b640 1 -- 172.21.15.49:0/1627273914 wait complete. 2024-03-26T18:00:07.434 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-26T18:00:07.434 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.567+0000 7f3d6559d640 1 -- 172.21.15.49:0/3349033663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d601006c0 msgr2=0x7f3d60100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.567+0000 7f3d6559d640 1 --2- 172.21.15.49:0/3349033663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d601006c0 0x7f3d60100ae0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009900 tx=0x7f3d5002bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.567+0000 7f3d6559d640 1 -- 172.21.15.49:0/3349033663 shutdown_connections 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.567+0000 7f3d6559d640 1 --2- 172.21.15.49:0/3349033663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d601018b0 0x7f3d60101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.567+0000 7f3d6559d640 1 --2- 172.21.15.49:0/3349033663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d60102270 0x7f3d60108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.567+0000 7f3d6559d640 1 --2- 172.21.15.49:0/3349033663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d601006c0 0x7f3d60100ae0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.567+0000 7f3d6559d640 1 -- 172.21.15.49:0/3349033663 >> 172.21.15.49:0/3349033663 conn(0x7f3d600fbc30 msgr2=0x7f3d600fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.567+0000 7f3d6559d640 1 -- 172.21.15.49:0/3349033663 shutdown_connections 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.567+0000 7f3d6559d640 1 -- 172.21.15.49:0/3349033663 wait complete. 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d6559d640 1 Processor -- start 2024-03-26T18:00:07.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d6559d640 1 -- start start 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d6559d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d601006c0 0x7f3d60104650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d6559d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d601018b0 0x7f3d60102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d6559d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d60102270 0x7f3d601031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d6559d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6010c340 con 0x7f3d601018b0 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d6559d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6010c1c0 con 0x7f3d601006c0 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d6559d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6010c4c0 con 0x7f3d60102270 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d5f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d601018b0 0x7f3d60102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d5ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d601006c0 0x7f3d60104650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d5f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d601018b0 0x7f3d60102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41600/0 (socket says 172.21.15.49:41600) 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d5f7fe640 1 -- 172.21.15.49:0/895659834 learned_addr learned my addr 172.21.15.49:0/895659834 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d64d9c640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d60102270 0x7f3d601031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:07.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d5ffff640 1 -- 172.21.15.49:0/895659834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d60102270 msgr2=0x7f3d601031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d5ffff640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d60102270 0x7f3d601031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d5ffff640 1 -- 172.21.15.49:0/895659834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d601018b0 msgr2=0x7f3d60102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d5ffff640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d601018b0 0x7f3d60102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d5ffff640 1 -- 172.21.15.49:0/895659834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d60103a60 con 0x7f3d601006c0 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.568+0000 7f3d5f7fe640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d601018b0 0x7f3d60102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.569+0000 7f3d5ffff640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d601006c0 0x7f3d60104650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3d5002c170 tx=0x7f3d5002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.569+0000 7f3d5d7fa640 1 -- 172.21.15.49:0/895659834 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50041070 con 0x7f3d601006c0 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.569+0000 7f3d5d7fa640 1 -- 172.21.15.49:0/895659834 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d50004450 con 0x7f3d601006c0 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.569+0000 7f3d6559d640 1 -- 172.21.15.49:0/895659834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d601a1bf0 con 0x7f3d601006c0 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.569+0000 7f3d5d7fa640 1 -- 172.21.15.49:0/895659834 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50003e50 con 0x7f3d601006c0 2024-03-26T18:00:07.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.569+0000 7f3d6559d640 1 -- 172.21.15.49:0/895659834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601a2110 con 0x7f3d601006c0 2024-03-26T18:00:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.570+0000 7f3d6559d640 1 -- 172.21.15.49:0/895659834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d601a22e0 con 0x7f3d601006c0 2024-03-26T18:00:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.571+0000 7f3d5d7fa640 1 -- 172.21.15.49:0/895659834 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f3d50044070 con 0x7f3d601006c0 2024-03-26T18:00:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.571+0000 7f3d5d7fa640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3d480a30c0 0x7f3d480a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.572+0000 7f3d5f7fe640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3d480a30c0 0x7f3d480a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:07.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.572+0000 7f3d5d7fa640 1 -- 172.21.15.49:0/895659834 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3d500f1200 con 0x7f3d601006c0 2024-03-26T18:00:07.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.572+0000 7f3d5f7fe640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3d480a30c0 0x7f3d480a5580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d601043c0 tx=0x7f3d54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:07.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.575+0000 7f3d5d7fa640 1 -- 172.21.15.49:0/895659834 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3d500bd840 con 0x7f3d601006c0 2024-03-26T18:00:07.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2182501733' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] up:active 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1627273914' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1627273914' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] up:active 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T18:00:07.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-03-26T18:00:07.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-mon[32331]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-26T18:00:07.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.785+0000 7f3d6559d640 1 -- 172.21.15.49:0/895659834 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3d600fde30 con 0x7f3d480a30c0 2024-03-26T18:00:07.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.811+0000 7f3d5d7fa640 1 -- 172.21.15.49:0/895659834 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d600fde30 con 0x7f3d480a30c0 2024-03-26T18:00:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 -- 172.21.15.49:0/895659834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3d480a30c0 msgr2=0x7f3d480a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3d480a30c0 0x7f3d480a5580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d601043c0 tx=0x7f3d54005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 -- 172.21.15.49:0/895659834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d601006c0 msgr2=0x7f3d60104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d601006c0 0x7f3d60104650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3d5002c170 tx=0x7f3d5002c650 comp rx=0 tx=0).stop 2024-03-26T18:00:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 -- 172.21.15.49:0/895659834 shutdown_connections 2024-03-26T18:00:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3d480a30c0 0x7f3d480a5580 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d60102270 0x7f3d601031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d601018b0 0x7f3d60102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 --2- 172.21.15.49:0/895659834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d601006c0 0x7f3d60104650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 -- 172.21.15.49:0/895659834 >> 172.21.15.49:0/895659834 conn(0x7f3d600fbc30 msgr2=0x7f3d600fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:07.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 -- 172.21.15.49:0/895659834 shutdown_connections 2024-03-26T18:00:07.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.813+0000 7f3d6559d640 1 -- 172.21.15.49:0/895659834 wait complete. 2024-03-26T18:00:07.808 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-26T18:00:07.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2182501733' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T18:00:07.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] up:active 2024-03-26T18:00:07.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T18:00:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-26T18:00:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-03-26T18:00:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-26T18:00:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1627273914' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T18:00:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1627273914' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T18:00:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] up:active 2024-03-26T18:00:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T18:00:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T18:00:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-03-26T18:00:07.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:07 smithi171 ceph-mon[36033]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-26T18:00:07.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.939+0000 7f6be27ff640 1 -- 172.21.15.49:0/3829673768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc1019b0 msgr2=0x7f6bdc101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.939+0000 7f6be27ff640 1 --2- 172.21.15.49:0/3829673768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc1019b0 0x7f6bdc101e30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009900 tx=0x7f6bd002bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.939+0000 7f6be27ff640 1 -- 172.21.15.49:0/3829673768 shutdown_connections 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.939+0000 7f6be27ff640 1 --2- 172.21.15.49:0/3829673768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc102370 0x7f6bdc108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.939+0000 7f6be27ff640 1 --2- 172.21.15.49:0/3829673768 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc1019b0 0x7f6bdc101e30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.939+0000 7f6be27ff640 1 --2- 172.21.15.49:0/3829673768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc1007c0 0x7f6bdc100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.939+0000 7f6be27ff640 1 -- 172.21.15.49:0/3829673768 >> 172.21.15.49:0/3829673768 conn(0x7f6bdc0fbd50 msgr2=0x7f6bdc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.939+0000 7f6be27ff640 1 -- 172.21.15.49:0/3829673768 shutdown_connections 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.939+0000 7f6be27ff640 1 -- 172.21.15.49:0/3829673768 wait complete. 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6be27ff640 1 Processor -- start 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6be27ff640 1 -- start start 2024-03-26T18:00:07.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6be27ff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1007c0 0x7f6bdc199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6be27ff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc1019b0 0x7f6bdc19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6be27ff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc102370 0x7f6bdc1a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6be27ff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc10c440 con 0x7f6bdc102370 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6be27ff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc10c2c0 con 0x7f6bdc1019b0 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6be27ff640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc10c5c0 con 0x7f6bdc1007c0 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc1019b0 0x7f6bdc19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6be0d9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc102370 0x7f6bdc1a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6bdbfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1007c0 0x7f6bdc199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc1019b0 0x7f6bdc19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:43572/0 (socket says 172.21.15.49:43572) 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6bdbfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1007c0 0x7f6bdc199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:57408/0 (socket says 172.21.15.49:57408) 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6bdb7fe640 1 -- 172.21.15.49:0/749655708 learned_addr learned my addr 172.21.15.49:0/749655708 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:07.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.940+0000 7f6be0d9c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc102370 0x7f6bdc1a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41644/0 (socket says 172.21.15.49:41644) 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6bdbfff640 1 -- 172.21.15.49:0/749655708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc1019b0 msgr2=0x7f6bdc19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6bdbfff640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc1019b0 0x7f6bdc19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6bdbfff640 1 -- 172.21.15.49:0/749655708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc102370 msgr2=0x7f6bdc1a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6bdbfff640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc102370 0x7f6bdc1a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6bdbfff640 1 -- 172.21.15.49:0/749655708 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc1a19f0 con 0x7f6bdc1007c0 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6be0d9c640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc102370 0x7f6bdc1a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6bdbfff640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1007c0 0x7f6bdc199e40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00e9c0 tx=0x7f6bcc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6bd97fa640 1 -- 172.21.15.49:0/749655708 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00ccc0 con 0x7f6bdc1007c0 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6bd97fa640 1 -- 172.21.15.49:0/749655708 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bcc0035f0 con 0x7f6bdc1007c0 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6be27ff640 1 -- 172.21.15.49:0/749655708 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc1a1ce0 con 0x7f6bdc1007c0 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6bd97fa640 1 -- 172.21.15.49:0/749655708 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00ce60 con 0x7f6bdc1007c0 2024-03-26T18:00:07.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.941+0000 7f6be27ff640 1 -- 172.21.15.49:0/749655708 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bdc1a21b0 con 0x7f6bdc1007c0 2024-03-26T18:00:07.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.942+0000 7f6be27ff640 1 -- 172.21.15.49:0/749655708 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bdc1a23e0 con 0x7f6bdc1007c0 2024-03-26T18:00:07.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.943+0000 7f6bd97fa640 1 -- 172.21.15.49:0/749655708 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f6bcc0037a0 con 0x7f6bdc1007c0 2024-03-26T18:00:07.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.944+0000 7f6bd97fa640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6bc40a2f20 0x7f6bc40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:07.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.944+0000 7f6bd97fa640 1 -- 172.21.15.49:0/749655708 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6bcc0cf970 con 0x7f6bdc1007c0 2024-03-26T18:00:07.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.944+0000 7f6bdb7fe640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6bc40a2f20 0x7f6bc40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:07.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.944+0000 7f6bdb7fe640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6bc40a2f20 0x7f6bc40a53e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0000c00 tx=0x7f6bd002c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:07.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:07.949+0000 7f6bd97fa640 1 -- 172.21.15.49:0/749655708 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6bcc09c560 con 0x7f6bdc1007c0 2024-03-26T18:00:07.967 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2182501733' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T18:00:07.967 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] up:active 2024-03-26T18:00:07.967 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T18:00:07.967 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-26T18:00:07.967 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-03-26T18:00:07.967 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-26T18:00:07.967 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1627273914' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T18:00:07.967 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1627273914' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T18:00:07.968 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] up:active 2024-03-26T18:00:07.968 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T18:00:07.968 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T18:00:07.968 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-03-26T18:00:07.968 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:07 smithi115 ceph-mon[36174]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-26T18:00:08.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.160+0000 7f6be27ff640 1 -- 172.21.15.49:0/749655708 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6bdc0fdf10 con 0x7f6bc40a2f20 2024-03-26T18:00:08.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.185+0000 7f6bd97fa640 1 -- 172.21.15.49:0/749655708 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6bdc0fdf10 con 0x7f6bc40a2f20 2024-03-26T18:00:08.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.187+0000 7f6be27ff640 1 -- 172.21.15.49:0/749655708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6bc40a2f20 msgr2=0x7f6bc40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.187+0000 7f6be27ff640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6bc40a2f20 0x7f6bc40a53e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0000c00 tx=0x7f6bd002c880 comp rx=0 tx=0).stop 2024-03-26T18:00:08.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.187+0000 7f6be27ff640 1 -- 172.21.15.49:0/749655708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1007c0 msgr2=0x7f6bdc199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.187+0000 7f6be27ff640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1007c0 0x7f6bdc199e40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00e9c0 tx=0x7f6bcc00ee80 comp rx=0 tx=0).stop 2024-03-26T18:00:08.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.188+0000 7f6be27ff640 1 -- 172.21.15.49:0/749655708 shutdown_connections 2024-03-26T18:00:08.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.188+0000 7f6be27ff640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6bc40a2f20 0x7f6bc40a53e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.188+0000 7f6be27ff640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6bdc102370 0x7f6bdc1a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.188+0000 7f6be27ff640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc1019b0 0x7f6bdc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.188+0000 7f6be27ff640 1 --2- 172.21.15.49:0/749655708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1007c0 0x7f6bdc199e40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.188+0000 7f6be27ff640 1 -- 172.21.15.49:0/749655708 >> 172.21.15.49:0/749655708 conn(0x7f6bdc0fbd50 msgr2=0x7f6bdc0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:08.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.188+0000 7f6be27ff640 1 -- 172.21.15.49:0/749655708 shutdown_connections 2024-03-26T18:00:08.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.188+0000 7f6be27ff640 1 -- 172.21.15.49:0/749655708 wait complete. 2024-03-26T18:00:08.181 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:00:07.790+0000 7f34d71d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:00:08.181 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:00:07.790+0000 7f34d71d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:00:08.181 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:00:07.790+0000 7f34d71d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:00:08.181 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:00:07.790+0000 7f34d71d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:00:08.181 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:00:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:00:07.790+0000 7f34d71d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:00:08.182 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-26T18:00:08.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.312+0000 7f9955844640 1 -- 172.21.15.49:0/1602579413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9950102390 msgr2=0x7f9950108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.312+0000 7f9955844640 1 --2- 172.21.15.49:0/1602579413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9950102390 0x7f9950108c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9948009ca0 tx=0x7f994802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.312+0000 7f9955844640 1 -- 172.21.15.49:0/1602579413 shutdown_connections 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.312+0000 7f9955844640 1 --2- 172.21.15.49:0/1602579413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9950102390 0x7f9950108c20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.312+0000 7f9955844640 1 --2- 172.21.15.49:0/1602579413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99501019d0 0x7f9950101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.312+0000 7f9955844640 1 --2- 172.21.15.49:0/1602579413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99501007e0 0x7f9950100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.312+0000 7f9955844640 1 -- 172.21.15.49:0/1602579413 >> 172.21.15.49:0/1602579413 conn(0x7f99500fbd70 msgr2=0x7f99500fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.312+0000 7f9955844640 1 -- 172.21.15.49:0/1602579413 shutdown_connections 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.312+0000 7f9955844640 1 -- 172.21.15.49:0/1602579413 wait complete. 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.312+0000 7f9955844640 1 Processor -- start 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f9955844640 1 -- start start 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f9955844640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99501007e0 0x7f9950197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f9955844640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99501019d0 0x7f9950198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:08.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f9955844640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9950102390 0x7f995019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f9955844640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f995010d580 con 0x7f99501007e0 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f9955844640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f995010d400 con 0x7f9950102390 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f9955844640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f995010d700 con 0x7f99501019d0 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99501007e0 0x7f9950197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9950102390 0x7f995019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99501007e0 0x7f9950197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41656/0 (socket says 172.21.15.49:41656) 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994e7fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99501019d0 0x7f9950198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994effd640 1 -- 172.21.15.49:0/3730578922 learned_addr learned my addr 172.21.15.49:0/3730578922 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994effd640 1 -- 172.21.15.49:0/3730578922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99501019d0 msgr2=0x7f9950198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994effd640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99501019d0 0x7f9950198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994effd640 1 -- 172.21.15.49:0/3730578922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9950102390 msgr2=0x7f995019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994effd640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9950102390 0x7f995019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994effd640 1 -- 172.21.15.49:0/3730578922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995019f7e0 con 0x7f99501007e0 2024-03-26T18:00:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.313+0000 7f994e7fc640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99501019d0 0x7f9950198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.314+0000 7f994effd640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99501007e0 0x7f9950197bc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f994400e9c0 tx=0x7f994400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.314+0000 7f9954842640 1 -- 172.21.15.49:0/3730578922 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994400cd80 con 0x7f99501007e0 2024-03-26T18:00:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.314+0000 7f9954842640 1 -- 172.21.15.49:0/3730578922 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99440035f0 con 0x7f99501007e0 2024-03-26T18:00:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.314+0000 7f9955844640 1 -- 172.21.15.49:0/3730578922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995019fad0 con 0x7f99501007e0 2024-03-26T18:00:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.314+0000 7f9954842640 1 -- 172.21.15.49:0/3730578922 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944010690 con 0x7f99501007e0 2024-03-26T18:00:08.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.314+0000 7f9955844640 1 -- 172.21.15.49:0/3730578922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995019ffd0 con 0x7f99501007e0 2024-03-26T18:00:08.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.315+0000 7f9955844640 1 -- 172.21.15.49:0/3730578922 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9950107aa0 con 0x7f99501007e0 2024-03-26T18:00:08.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.316+0000 7f9954842640 1 -- 172.21.15.49:0/3730578922 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f99440037a0 con 0x7f99501007e0 2024-03-26T18:00:08.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.316+0000 7f9954842640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99380a2f20 0x7f99380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:08.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.317+0000 7f994e7fc640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99380a2f20 0x7f99380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:08.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.317+0000 7f9954842640 1 -- 172.21.15.49:0/3730578922 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f99440cf990 con 0x7f99501007e0 2024-03-26T18:00:08.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.317+0000 7f994e7fc640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99380a2f20 0x7f99380a53e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f99501990d0 tx=0x7f9940005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:08.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.321+0000 7f9954842640 1 -- 172.21.15.49:0/3730578922 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f994409c580 con 0x7f99501007e0 2024-03-26T18:00:08.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.542+0000 7f9955844640 1 -- 172.21.15.49:0/3730578922 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9950061d00 con 0x7f99380a2f20 2024-03-26T18:00:08.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.547+0000 7f9954842640 1 -- 172.21.15.49:0/3730578922 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9950061d00 con 0x7f99380a2f20 2024-03-26T18:00:08.531 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/qa/sv_0/e2f188f1-77a1-4347-911b-2846f372e30d 2024-03-26T18:00:08.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.549+0000 7f9955844640 1 -- 172.21.15.49:0/3730578922 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99380a2f20 msgr2=0x7f99380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.549+0000 7f9955844640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99380a2f20 0x7f99380a53e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f99501990d0 tx=0x7f9940005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:08.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.549+0000 7f9955844640 1 -- 172.21.15.49:0/3730578922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99501007e0 msgr2=0x7f9950197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.549+0000 7f9955844640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99501007e0 0x7f9950197bc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f994400e9c0 tx=0x7f994400ee80 comp rx=0 tx=0).stop 2024-03-26T18:00:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.550+0000 7f9955844640 1 -- 172.21.15.49:0/3730578922 shutdown_connections 2024-03-26T18:00:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.550+0000 7f9955844640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f99380a2f20 0x7f99380a53e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.550+0000 7f9955844640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99501019d0 0x7f9950198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.550+0000 7f9955844640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9950102390 0x7f995019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.550+0000 7f9955844640 1 --2- 172.21.15.49:0/3730578922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99501007e0 0x7f9950197bc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.550+0000 7f9955844640 1 -- 172.21.15.49:0/3730578922 >> 172.21.15.49:0/3730578922 conn(0x7f99500fbd70 msgr2=0x7f9950104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.550+0000 7f9955844640 1 -- 172.21.15.49:0/3730578922 shutdown_connections 2024-03-26T18:00:08.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.550+0000 7f9955844640 1 -- 172.21.15.49:0/3730578922 wait complete. 2024-03-26T18:00:08.544 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-26T18:00:08.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 -- 172.21.15.49:0/1757880584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b6c1014b0 msgr2=0x7f6b6c101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 --2- 172.21.15.49:0/1757880584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b6c1014b0 0x7f6b6c101930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009ca0 tx=0x7f6b6402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 -- 172.21.15.49:0/1757880584 shutdown_connections 2024-03-26T18:00:08.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 --2- 172.21.15.49:0/1757880584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b6c1014b0 0x7f6b6c101930 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 --2- 172.21.15.49:0/1757880584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b6c100b10 0x7f6b6c100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 --2- 172.21.15.49:0/1757880584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b6c1001b0 0x7f6b6c1005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 -- 172.21.15.49:0/1757880584 >> 172.21.15.49:0/1757880584 conn(0x7f6b6c0fbd70 msgr2=0x7f6b6c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 -- 172.21.15.49:0/1757880584 shutdown_connections 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 -- 172.21.15.49:0/1757880584 wait complete. 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 Processor -- start 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.675+0000 7f6b713b3640 1 -- start start 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b713b3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b6c1001b0 0x7f6b6c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b713b3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b6c100b10 0x7f6b6c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b713b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b6c1014b0 0x7f6b6c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b713b3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10f750 con 0x7f6b6c1014b0 2024-03-26T18:00:08.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b713b3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10f5d0 con 0x7f6b6c1001b0 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b713b3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10f8d0 con 0x7f6b6c100b10 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b6b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b6c1014b0 0x7f6b6c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b6b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b6c1014b0 0x7f6b6c1a1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45586/0 (socket says 172.21.15.49:45586) 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b6affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b6c1001b0 0x7f6b6c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b6b7fe640 1 -- 172.21.15.49:0/592032389 learned_addr learned my addr 172.21.15.49:0/592032389 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b63fff640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b6c100b10 0x7f6b6c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b6b7fe640 1 -- 172.21.15.49:0/592032389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b6c100b10 msgr2=0x7f6b6c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b6b7fe640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b6c100b10 0x7f6b6c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b6b7fe640 1 -- 172.21.15.49:0/592032389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b6c1001b0 msgr2=0x7f6b6c199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b6b7fe640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b6c1001b0 0x7f6b6c199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b6b7fe640 1 -- 172.21.15.49:0/592032389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c1a19d0 con 0x7f6b6c1014b0 2024-03-26T18:00:08.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b6b7fe640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b6c1014b0 0x7f6b6c1a1330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b6402c630 tx=0x7f6b64032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:08.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b68ff9640 1 -- 172.21.15.49:0/592032389 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64041020 con 0x7f6b6c1014b0 2024-03-26T18:00:08.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b68ff9640 1 -- 172.21.15.49:0/592032389 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b64034cd0 con 0x7f6b6c1014b0 2024-03-26T18:00:08.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b713b3640 1 -- 172.21.15.49:0/592032389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c1a1c60 con 0x7f6b6c1014b0 2024-03-26T18:00:08.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b68ff9640 1 -- 172.21.15.49:0/592032389 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64039040 con 0x7f6b6c1014b0 2024-03-26T18:00:08.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.676+0000 7f6b713b3640 1 -- 172.21.15.49:0/592032389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c1a21a0 con 0x7f6b6c1014b0 2024-03-26T18:00:08.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.678+0000 7f6b713b3640 1 -- 172.21.15.49:0/592032389 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c10af00 con 0x7f6b6c1014b0 2024-03-26T18:00:08.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.678+0000 7f6b68ff9640 1 -- 172.21.15.49:0/592032389 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f6b64044020 con 0x7f6b6c1014b0 2024-03-26T18:00:08.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.679+0000 7f6b68ff9640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b580a2fa0 0x7f6b580a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:08.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.679+0000 7f6b6affd640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b580a2fa0 0x7f6b580a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:08.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.680+0000 7f6b68ff9640 1 -- 172.21.15.49:0/592032389 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6b640f1450 con 0x7f6b6c1014b0 2024-03-26T18:00:08.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.680+0000 7f6b6affd640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b580a2fa0 0x7f6b580a5460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c19b290 tx=0x7f6b5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:08.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.684+0000 7f6b68ff9640 1 -- 172.21.15.49:0/592032389 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6b6404b020 con 0x7f6b6c1014b0 2024-03-26T18:00:08.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:08 smithi049 ceph-mon[32331]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-03-26T18:00:08.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:08 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] up:active 2024-03-26T18:00:08.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:08 smithi049 ceph-mon[32331]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T18:00:08.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:08 smithi049 ceph-mon[32331]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T18:00:08.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:08 smithi049 ceph-mon[32331]: Cluster is now healthy 2024-03-26T18:00:08.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:08 smithi049 ceph-mon[32331]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-03-26T18:00:08.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:08 smithi049 ceph-mon[32331]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-26T18:00:08.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:08 smithi049 ceph-mon[32331]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-26T18:00:08.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:08 smithi171 ceph-mon[36033]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-03-26T18:00:08.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:08 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] up:active 2024-03-26T18:00:08.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:08 smithi171 ceph-mon[36033]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T18:00:08.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:08 smithi171 ceph-mon[36033]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T18:00:08.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:08 smithi171 ceph-mon[36033]: Cluster is now healthy 2024-03-26T18:00:08.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:08 smithi171 ceph-mon[36033]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-03-26T18:00:08.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:08 smithi171 ceph-mon[36033]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-26T18:00:08.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:08 smithi171 ceph-mon[36033]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-26T18:00:08.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.897+0000 7f6b713b3640 1 -- 172.21.15.49:0/592032389 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6b6c061e50 con 0x7f6b580a2fa0 2024-03-26T18:00:08.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.924+0000 7f6b68ff9640 1 -- 172.21.15.49:0/592032389 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6b6c061e50 con 0x7f6b580a2fa0 2024-03-26T18:00:08.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 -- 172.21.15.49:0/592032389 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b580a2fa0 msgr2=0x7f6b580a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b580a2fa0 0x7f6b580a5460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c19b290 tx=0x7f6b5c005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:08.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 -- 172.21.15.49:0/592032389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b6c1014b0 msgr2=0x7f6b6c1a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:08.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b6c1014b0 0x7f6b6c1a1330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b6402c630 tx=0x7f6b64032ea0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 -- 172.21.15.49:0/592032389 shutdown_connections 2024-03-26T18:00:08.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b580a2fa0 0x7f6b580a5460 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b6c100b10 0x7f6b6c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b6c1014b0 0x7f6b6c1a1330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 --2- 172.21.15.49:0/592032389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b6c1001b0 0x7f6b6c199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:08.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 -- 172.21.15.49:0/592032389 >> 172.21.15.49:0/592032389 conn(0x7f6b6c0fbd70 msgr2=0x7f6b6c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:08.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 -- 172.21.15.49:0/592032389 shutdown_connections 2024-03-26T18:00:08.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:08.926+0000 7f6b713b3640 1 -- 172.21.15.49:0/592032389 wait complete. 2024-03-26T18:00:08.921 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-26T18:00:08.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:08 smithi115 ceph-mon[36174]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-03-26T18:00:08.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:08 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] up:active 2024-03-26T18:00:08.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:08 smithi115 ceph-mon[36174]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T18:00:08.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:08 smithi115 ceph-mon[36174]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T18:00:08.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:08 smithi115 ceph-mon[36174]: Cluster is now healthy 2024-03-26T18:00:08.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:08 smithi115 ceph-mon[36174]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-03-26T18:00:08.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:08 smithi115 ceph-mon[36174]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-03-26T18:00:08.989 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:08 smithi115 ceph-mon[36174]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-26T18:00:09.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 -- 172.21.15.49:0/4115404854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd228103480 msgr2=0x7fd228109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 --2- 172.21.15.49:0/4115404854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd228103480 0x7fd228109d10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd218009ca0 tx=0x7fd21802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 -- 172.21.15.49:0/4115404854 shutdown_connections 2024-03-26T18:00:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 --2- 172.21.15.49:0/4115404854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd228103480 0x7fd228109d10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 --2- 172.21.15.49:0/4115404854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd228102ac0 0x7fd228102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 --2- 172.21.15.49:0/4115404854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2281018d0 0x7fd228101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 -- 172.21.15.49:0/4115404854 >> 172.21.15.49:0/4115404854 conn(0x7fd2280fce40 msgr2=0x7fd2280ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:09.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 -- 172.21.15.49:0/4115404854 shutdown_connections 2024-03-26T18:00:09.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 -- 172.21.15.49:0/4115404854 wait complete. 2024-03-26T18:00:09.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 Processor -- start 2024-03-26T18:00:09.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.051+0000 7fd22d4f8640 1 -- start start 2024-03-26T18:00:09.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd22d4f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2281018d0 0x7fd228197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd22d4f8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd228102ac0 0x7fd2281980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd22d4f8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd228103480 0x7fd22819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd22d4f8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd22810d530 con 0x7fd2281018d0 2024-03-26T18:00:09.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd22d4f8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd22810d3b0 con 0x7fd228102ac0 2024-03-26T18:00:09.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd22d4f8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd22810d6b0 con 0x7fd228103480 2024-03-26T18:00:09.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd226ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2281018d0 0x7fd228197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd226ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2281018d0 0x7fd228197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45616/0 (socket says 172.21.15.49:45616) 2024-03-26T18:00:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd226ffd640 1 -- 172.21.15.49:0/768831652 learned_addr learned my addr 172.21.15.49:0/768831652 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd2267fc640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd228102ac0 0x7fd2281980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd2277fe640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd228103480 0x7fd22819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd226ffd640 1 -- 172.21.15.49:0/768831652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd228103480 msgr2=0x7fd22819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd226ffd640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd228103480 0x7fd22819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd226ffd640 1 -- 172.21.15.49:0/768831652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd228102ac0 msgr2=0x7fd2281980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd226ffd640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd228102ac0 0x7fd2281980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd226ffd640 1 -- 172.21.15.49:0/768831652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22819f790 con 0x7fd2281018d0 2024-03-26T18:00:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd2277fe640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd228103480 0x7fd22819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:00:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd226ffd640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2281018d0 0x7fd228197b70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd22000e9c0 tx=0x7fd22000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd21ffff640 1 -- 172.21.15.49:0/768831652 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22000cde0 con 0x7fd2281018d0 2024-03-26T18:00:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd21ffff640 1 -- 172.21.15.49:0/768831652 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd220010910 con 0x7fd2281018d0 2024-03-26T18:00:09.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd22d4f8640 1 -- 172.21.15.49:0/768831652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22819fa80 con 0x7fd2281018d0 2024-03-26T18:00:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd21ffff640 1 -- 172.21.15.49:0/768831652 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220018a00 con 0x7fd2281018d0 2024-03-26T18:00:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.052+0000 7fd22d4f8640 1 -- 172.21.15.49:0/768831652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22819ffd0 con 0x7fd2281018d0 2024-03-26T18:00:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.053+0000 7fd22d4f8640 1 -- 172.21.15.49:0/768831652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd228108c40 con 0x7fd2281018d0 2024-03-26T18:00:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.055+0000 7fd21ffff640 1 -- 172.21.15.49:0/768831652 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7fd22001f020 con 0x7fd2281018d0 2024-03-26T18:00:09.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.055+0000 7fd21ffff640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd2140a2f20 0x7fd2140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.055+0000 7fd2267fc640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd2140a2f20 0x7fd2140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.056+0000 7fd21ffff640 1 -- 172.21.15.49:0/768831652 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd2200cf8b0 con 0x7fd2281018d0 2024-03-26T18:00:09.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.056+0000 7fd2267fc640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd2140a2f20 0x7fd2140a53e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd228199080 tx=0x7fd210005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:09.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.060+0000 7fd21ffff640 1 -- 172.21.15.49:0/768831652 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd22009c4a0 con 0x7fd2281018d0 2024-03-26T18:00:09.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.278+0000 7fd22d4f8640 1 -- 172.21.15.49:0/768831652 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd228061d90 con 0x7fd2140a2f20 2024-03-26T18:00:09.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.282+0000 7fd21ffff640 1 -- 172.21.15.49:0/768831652 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd228061d90 con 0x7fd2140a2f20 2024-03-26T18:00:09.266 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/qa/sv_1/af486460-0fa8-40db-9e10-2412dd703b0d 2024-03-26T18:00:09.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 -- 172.21.15.49:0/768831652 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd2140a2f20 msgr2=0x7fd2140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd2140a2f20 0x7fd2140a53e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd228199080 tx=0x7fd210005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:09.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 -- 172.21.15.49:0/768831652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2281018d0 msgr2=0x7fd228197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2281018d0 0x7fd228197b70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd22000e9c0 tx=0x7fd22000ee80 comp rx=0 tx=0).stop 2024-03-26T18:00:09.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 -- 172.21.15.49:0/768831652 shutdown_connections 2024-03-26T18:00:09.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd2140a2f20 0x7fd2140a53e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd228103480 0x7fd22819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd228102ac0 0x7fd2281980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 --2- 172.21.15.49:0/768831652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2281018d0 0x7fd228197b70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 -- 172.21.15.49:0/768831652 >> 172.21.15.49:0/768831652 conn(0x7fd2280fce40 msgr2=0x7fd228105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:09.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 -- 172.21.15.49:0/768831652 shutdown_connections 2024-03-26T18:00:09.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.285+0000 7fd22d4f8640 1 -- 172.21.15.49:0/768831652 wait complete. 2024-03-26T18:00:09.281 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:00:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 -- 172.21.15.49:0/71683031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8318102410 msgr2=0x7f8318108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 --2- 172.21.15.49:0/71683031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8318102410 0x7f8318108ca0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8310009be0 tx=0x7f831002bef0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 -- 172.21.15.49:0/71683031 shutdown_connections 2024-03-26T18:00:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 --2- 172.21.15.49:0/71683031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8318102410 0x7f8318108ca0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 --2- 172.21.15.49:0/71683031 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8318101a50 0x7f8318101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 --2- 172.21.15.49:0/71683031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8318100860 0x7f8318100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 -- 172.21.15.49:0/71683031 >> 172.21.15.49:0/71683031 conn(0x7f83180fbdf0 msgr2=0x7f83180fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 -- 172.21.15.49:0/71683031 shutdown_connections 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 -- 172.21.15.49:0/71683031 wait complete. 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 Processor -- start 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 -- start start 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8318100860 0x7f8318197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8318101a50 0x7f83181980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8318102410 0x7f831819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f831810c470 con 0x7f8318102410 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f831810c2f0 con 0x7f8318100860 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.406+0000 7f831f787640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f831810c5f0 con 0x7f8318101a50 2024-03-26T18:00:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831d523640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8318100860 0x7f8318197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831dd24640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8318102410 0x7f831819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831d523640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8318100860 0x7f8318197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:35056/0 (socket says 172.21.15.49:35056) 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831d523640 1 -- 172.21.15.49:0/479878660 learned_addr learned my addr 172.21.15.49:0/479878660 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831cd22640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8318101a50 0x7f83181980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831d523640 1 -- 172.21.15.49:0/479878660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8318101a50 msgr2=0x7f83181980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831d523640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8318101a50 0x7f83181980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831d523640 1 -- 172.21.15.49:0/479878660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8318102410 msgr2=0x7f831819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831d523640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8318102410 0x7f831819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831d523640 1 -- 172.21.15.49:0/479878660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831819f7c0 con 0x7f8318100860 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831dd24640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8318102410 0x7f831819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831d523640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8318100860 0x7f8318197ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f831400e9c0 tx=0x7f831400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f830e7fc640 1 -- 172.21.15.49:0/479878660 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831400cd80 con 0x7f8318100860 2024-03-26T18:00:09.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831f787640 1 -- 172.21.15.49:0/479878660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831819fab0 con 0x7f8318100860 2024-03-26T18:00:09.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f830e7fc640 1 -- 172.21.15.49:0/479878660 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83140035f0 con 0x7f8318100860 2024-03-26T18:00:09.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f830e7fc640 1 -- 172.21.15.49:0/479878660 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314010690 con 0x7f8318100860 2024-03-26T18:00:09.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.407+0000 7f831f787640 1 -- 172.21.15.49:0/479878660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831819ffb0 con 0x7f8318100860 2024-03-26T18:00:09.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.408+0000 7f831f787640 1 -- 172.21.15.49:0/479878660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83181a0240 con 0x7f8318100860 2024-03-26T18:00:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.410+0000 7f830e7fc640 1 -- 172.21.15.49:0/479878660 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) ==== 137854+0+0 (secure 0 0 0) 0x7f83140037a0 con 0x7f8318100860 2024-03-26T18:00:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.410+0000 7f830e7fc640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f83040a2f20 0x7f83040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.410+0000 7f831cd22640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f83040a2f20 0x7f83040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.410+0000 7f830e7fc640 1 -- 172.21.15.49:0/479878660 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f83140cfa30 con 0x7f8318100860 2024-03-26T18:00:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.411+0000 7f831cd22640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f83040a2f20 0x7f83040a53e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f83181990b0 tx=0x7f8308009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.418+0000 7f830e7fc640 1 -- 172.21.15.49:0/479878660 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f831409c620 con 0x7f8318100860 2024-03-26T18:00:09.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.529+0000 7f830e7fc640 1 -- 172.21.15.49:0/479878660 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f8314098590 con 0x7f8318100860 2024-03-26T18:00:09.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.656+0000 7f831f787640 1 -- 172.21.15.49:0/479878660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83181a0910 con 0x7f8318100860 2024-03-26T18:00:09.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.657+0000 7f830e7fc640 1 -- 172.21.15.49:0/479878660 <== mon.2 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11943 (secure 0 0 0) 0x7f83140243c0 con 0x7f8318100860 2024-03-26T18:00:09.641 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:00:09.641 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:59:46.796381+0000","modified":"2024-03-26T18:00:09.450708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24503,"mds_2":24478,"mds_3":24428,"mds_4":24388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24478,24503,24388,24428,24533]},"id":1}]} 2024-03-26T18:00:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-03-26T18:00:09.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 -- 172.21.15.49:0/479878660 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f83040a2f20 msgr2=0x7f83040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f83040a2f20 0x7f83040a53e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f83181990b0 tx=0x7f8308009260 comp rx=0 tx=0).stop 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 -- 172.21.15.49:0/479878660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8318100860 msgr2=0x7f8318197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8318100860 0x7f8318197ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f831400e9c0 tx=0x7f831400ee80 comp rx=0 tx=0).stop 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 -- 172.21.15.49:0/479878660 shutdown_connections 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f83040a2f20 0x7f83040a53e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8318101a50 0x7f83181980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8318102410 0x7f831819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 --2- 172.21.15.49:0/479878660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8318100860 0x7f8318197ba0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 -- 172.21.15.49:0/479878660 >> 172.21.15.49:0/479878660 conn(0x7f83180fbdf0 msgr2=0x7f83180fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 -- 172.21.15.49:0/479878660 shutdown_connections 2024-03-26T18:00:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.659+0000 7f831f787640 1 -- 172.21.15.49:0/479878660 wait complete. 2024-03-26T18:00:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:09 smithi049 ceph-mon[32331]: from='client.24773 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:09 smithi049 ceph-mon[32331]: from='client.24664 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:09 smithi049 ceph-mon[32331]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:09 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] up:active 2024-03-26T18:00:09.652 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:09 smithi049 ceph-mon[32331]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-26T18:00:09.653 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:09 smithi049 ceph-mon[32331]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:00:09.654 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:00:09.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.779+0000 7f02c206c640 1 -- 172.21.15.49:0/1437610146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02bc1034a0 msgr2=0x7f02bc109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.779+0000 7f02c206c640 1 --2- 172.21.15.49:0/1437610146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02bc1034a0 0x7f02bc109d30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009ca0 tx=0x7f02b402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.779+0000 7f02c206c640 1 -- 172.21.15.49:0/1437610146 shutdown_connections 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.779+0000 7f02c206c640 1 --2- 172.21.15.49:0/1437610146 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02bc1034a0 0x7f02bc109d30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.779+0000 7f02c206c640 1 --2- 172.21.15.49:0/1437610146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02bc102ae0 0x7f02bc102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.779+0000 7f02c206c640 1 --2- 172.21.15.49:0/1437610146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc1018f0 0x7f02bc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.779+0000 7f02c206c640 1 -- 172.21.15.49:0/1437610146 >> 172.21.15.49:0/1437610146 conn(0x7f02bc0fcea0 msgr2=0x7f02bc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.779+0000 7f02c206c640 1 -- 172.21.15.49:0/1437610146 shutdown_connections 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.779+0000 7f02c206c640 1 -- 172.21.15.49:0/1437610146 wait complete. 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02c206c640 1 Processor -- start 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02c206c640 1 -- start start 2024-03-26T18:00:09.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02c206c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02bc1018f0 0x7f02bc197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02c206c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02bc102ae0 0x7f02bc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02c206c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc1034a0 0x7f02bc19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02c206c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc10d550 con 0x7f02bc102ae0 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02c206c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc10d3d0 con 0x7f02bc1018f0 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02c206c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc10d6d0 con 0x7f02bc1034a0 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02baffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02bc102ae0 0x7f02bc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02bc1018f0 0x7f02bc197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02baffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02bc102ae0 0x7f02bc1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45654/0 (socket says 172.21.15.49:45654) 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02baffd640 1 -- 172.21.15.49:0/2080088516 learned_addr learned my addr 172.21.15.49:0/2080088516 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02bbfff640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc1034a0 0x7f02bc19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02baffd640 1 -- 172.21.15.49:0/2080088516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc1034a0 msgr2=0x7f02bc19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02baffd640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc1034a0 0x7f02bc19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02baffd640 1 -- 172.21.15.49:0/2080088516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02bc1018f0 msgr2=0x7f02bc197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02baffd640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02bc1018f0 0x7f02bc197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02baffd640 1 -- 172.21.15.49:0/2080088516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc19f7b0 con 0x7f02bc102ae0 2024-03-26T18:00:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.780+0000 7f02bb7fe640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02bc1018f0 0x7f02bc197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.781+0000 7f02baffd640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02bc102ae0 0x7f02bc1980d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f02b000e9f0 tx=0x7f02b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.781+0000 7f02b8ff9640 1 -- 172.21.15.49:0/2080088516 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b000cd80 con 0x7f02bc102ae0 2024-03-26T18:00:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.781+0000 7f02b8ff9640 1 -- 172.21.15.49:0/2080088516 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b0010910 con 0x7f02bc102ae0 2024-03-26T18:00:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.781+0000 7f02c206c640 1 -- 172.21.15.49:0/2080088516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02bc19faa0 con 0x7f02bc102ae0 2024-03-26T18:00:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.781+0000 7f02b8ff9640 1 -- 172.21.15.49:0/2080088516 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0018990 con 0x7f02bc102ae0 2024-03-26T18:00:09.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.781+0000 7f02c206c640 1 -- 172.21.15.49:0/2080088516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02bc19fff0 con 0x7f02bc102ae0 2024-03-26T18:00:09.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.782+0000 7f02c206c640 1 -- 172.21.15.49:0/2080088516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02bc108c10 con 0x7f02bc102ae0 2024-03-26T18:00:09.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.783+0000 7f02b8ff9640 1 -- 172.21.15.49:0/2080088516 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f02b0010ac0 con 0x7f02bc102ae0 2024-03-26T18:00:09.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.783+0000 7f02b8ff9640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f02a40a2fb0 0x7f02a40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:09.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.784+0000 7f02bb7fe640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f02a40a2fb0 0x7f02a40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:09.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.784+0000 7f02b8ff9640 1 -- 172.21.15.49:0/2080088516 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f02b00cfac0 con 0x7f02bc102ae0 2024-03-26T18:00:09.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.784+0000 7f02bb7fe640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f02a40a2fb0 0x7f02a40a5470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f02bc102940 tx=0x7f02ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:09.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.788+0000 7f02b8ff9640 1 -- 172.21.15.49:0/2080088516 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f02b009bf00 con 0x7f02bc102ae0 2024-03-26T18:00:09.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:09 smithi171 ceph-mon[36033]: from='client.24773 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:09.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:09 smithi171 ceph-mon[36033]: from='client.24664 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:09.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:09 smithi171 ceph-mon[36033]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:09.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:09 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] up:active 2024-03-26T18:00:09.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:09 smithi171 ceph-mon[36033]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-26T18:00:09.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:09 smithi171 ceph-mon[36033]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:00:09.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.974+0000 7f02c206c640 1 -- 172.21.15.49:0/2080088516 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02bc02d440 con 0x7f02bc102ae0 2024-03-26T18:00:09.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.975+0000 7f02b8ff9640 1 -- 172.21.15.49:0/2080088516 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24136 (secure 0 0 0) 0x7f02b009ffa0 con 0x7f02bc102ae0 2024-03-26T18:00:09.959 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:00:09.959 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:51.836345+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:00:09.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 -- 172.21.15.49:0/2080088516 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f02a40a2fb0 msgr2=0x7f02a40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f02a40a2fb0 0x7f02a40a5470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f02bc102940 tx=0x7f02ac005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:09.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 -- 172.21.15.49:0/2080088516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02bc102ae0 msgr2=0x7f02bc1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02bc102ae0 0x7f02bc1980d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f02b000e9f0 tx=0x7f02b000eeb0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 -- 172.21.15.49:0/2080088516 shutdown_connections 2024-03-26T18:00:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f02a40a2fb0 0x7f02a40a5470 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02bc1034a0 0x7f02bc19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02bc102ae0 0x7f02bc1980d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 --2- 172.21.15.49:0/2080088516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02bc1018f0 0x7f02bc197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 -- 172.21.15.49:0/2080088516 >> 172.21.15.49:0/2080088516 conn(0x7f02bc0fcea0 msgr2=0x7f02bc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 -- 172.21.15.49:0/2080088516 shutdown_connections 2024-03-26T18:00:09.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:09.978+0000 7f02c206c640 1 -- 172.21.15.49:0/2080088516 wait complete. 2024-03-26T18:00:09.973 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-26T18:00:09.973 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T18:00:09.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:09 smithi115 ceph-mon[36174]: from='client.24773 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:09.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:09 smithi115 ceph-mon[36174]: from='client.24664 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:09.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:09 smithi115 ceph-mon[36174]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:09.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:09 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] up:active 2024-03-26T18:00:09.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:09 smithi115 ceph-mon[36174]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-03-26T18:00:09.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:09 smithi115 ceph-mon[36174]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:00:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.098+0000 7f74b44cc640 1 -- 172.21.15.49:0/2573902954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ac1019d0 msgr2=0x7f74ac101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.098+0000 7f74b44cc640 1 --2- 172.21.15.49:0/2573902954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ac1019d0 0x7f74ac101e50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f749c009900 tx=0x7f749c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 -- 172.21.15.49:0/2573902954 shutdown_connections 2024-03-26T18:00:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 --2- 172.21.15.49:0/2573902954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac102390 0x7f74ac108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 --2- 172.21.15.49:0/2573902954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ac1019d0 0x7f74ac101e50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 --2- 172.21.15.49:0/2573902954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74ac1007e0 0x7f74ac100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 -- 172.21.15.49:0/2573902954 >> 172.21.15.49:0/2573902954 conn(0x7f74ac0fbd50 msgr2=0x7f74ac0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 -- 172.21.15.49:0/2573902954 shutdown_connections 2024-03-26T18:00:10.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 -- 172.21.15.49:0/2573902954 wait complete. 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 Processor -- start 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 -- start start 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74ac1007e0 0x7f74ac197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ac1019d0 0x7f74ac198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac102390 0x7f74ac19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac10d580 con 0x7f74ac1019d0 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac10d400 con 0x7f74ac102390 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b44cc640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac10d700 con 0x7f74ac1007e0 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b1a67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ac1019d0 0x7f74ac198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b1a67640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ac1019d0 0x7f74ac198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45668/0 (socket says 172.21.15.49:45668) 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b2a69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac102390 0x7f74ac19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b1a67640 1 -- 172.21.15.49:0/2839587263 learned_addr learned my addr 172.21.15.49:0/2839587263 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:10.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.099+0000 7f74b2268640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74ac1007e0 0x7f74ac197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:10.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74b1a67640 1 -- 172.21.15.49:0/2839587263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74ac1007e0 msgr2=0x7f74ac197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:10.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74b1a67640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74ac1007e0 0x7f74ac197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74b1a67640 1 -- 172.21.15.49:0/2839587263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac102390 msgr2=0x7f74ac19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:10.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74b1a67640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac102390 0x7f74ac19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74b1a67640 1 -- 172.21.15.49:0/2839587263 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac19f850 con 0x7f74ac1019d0 2024-03-26T18:00:10.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74b2a69640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac102390 0x7f74ac19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:10.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74b1a67640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ac1019d0 0x7f74ac198180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f749c009230 tx=0x7f749c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:10.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74a37fe640 1 -- 172.21.15.49:0/2839587263 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c041070 con 0x7f74ac1019d0 2024-03-26T18:00:10.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74b44cc640 1 -- 172.21.15.49:0/2839587263 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ac19fae0 con 0x7f74ac1019d0 2024-03-26T18:00:10.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74a37fe640 1 -- 172.21.15.49:0/2839587263 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f749c004450 con 0x7f74ac1019d0 2024-03-26T18:00:10.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74a37fe640 1 -- 172.21.15.49:0/2839587263 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c003db0 con 0x7f74ac1019d0 2024-03-26T18:00:10.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.100+0000 7f74b44cc640 1 -- 172.21.15.49:0/2839587263 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ac1a0020 con 0x7f74ac1019d0 2024-03-26T18:00:10.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.101+0000 7f74b44cc640 1 -- 172.21.15.49:0/2839587263 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ac107bd0 con 0x7f74ac1019d0 2024-03-26T18:00:10.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.102+0000 7f74a37fe640 1 -- 172.21.15.49:0/2839587263 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f749c044070 con 0x7f74ac1019d0 2024-03-26T18:00:10.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.103+0000 7f74a37fe640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f74980a2fb0 0x7f74980a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:10.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.103+0000 7f74b2268640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f74980a2fb0 0x7f74980a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:10.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.103+0000 7f74a37fe640 1 -- 172.21.15.49:0/2839587263 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f749c0f1250 con 0x7f74ac1019d0 2024-03-26T18:00:10.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.103+0000 7f74b2268640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f74980a2fb0 0x7f74980a5470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f74ac199110 tx=0x7f74a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:10.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.107+0000 7f74a37fe640 1 -- 172.21.15.49:0/2839587263 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f749c0bddf0 con 0x7f74ac1019d0 2024-03-26T18:00:10.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.288+0000 7f74b44cc640 1 -- 172.21.15.49:0/2839587263 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74ac061d00 con 0x7f74980a2fb0 2024-03-26T18:00:10.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.299+0000 7f74a37fe640 1 -- 172.21.15.49:0/2839587263 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340512 (secure 0 0 0) 0x7f74ac061d00 con 0x7f74980a2fb0 2024-03-26T18:00:10.287 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:00:10.289 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-26T18:00:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 -- 172.21.15.49:0/2839587263 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f74980a2fb0 msgr2=0x7f74980a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f74980a2fb0 0x7f74980a5470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f74ac199110 tx=0x7f74a8005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 -- 172.21.15.49:0/2839587263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ac1019d0 msgr2=0x7f74ac198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ac1019d0 0x7f74ac198180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f749c009230 tx=0x7f749c02c650 comp rx=0 tx=0).stop 2024-03-26T18:00:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 -- 172.21.15.49:0/2839587263 shutdown_connections 2024-03-26T18:00:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f74980a2fb0 0x7f74980a5470 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac102390 0x7f74ac19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74ac1019d0 0x7f74ac198180 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 --2- 172.21.15.49:0/2839587263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74ac1007e0 0x7f74ac197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 -- 172.21.15.49:0/2839587263 >> 172.21.15.49:0/2839587263 conn(0x7f74ac0fbd50 msgr2=0x7f74ac104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:10.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 -- 172.21.15.49:0/2839587263 shutdown_connections 2024-03-26T18:00:10.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.302+0000 7f74b44cc640 1 -- 172.21.15.49:0/2839587263 wait complete. 2024-03-26T18:00:10.297 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-03-26T18:00:08.707371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595944,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":137,"num_read_kb":120,"num_write":273,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360844,"kb_used_data":11664,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580980,"statfs":{"total":1151940427776,"available":1151570923520,"internally_reserved":0,"allocated":11943936,"data_stored":7551016,"data_compressed":285108,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3166,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002952"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725719+0000","last_change":"2024-03-26T17:59:47.232887+0000","last_active":"2024-03-26T17:59:52.725719+0000","last_peered":"2024-03-26T17:59:52.725719+0000","last_clean":"2024-03-26T17:59:52.725719+0000","last_became_active":"2024-03-26T17:59:47.232559+0000","last_became_peered":"2024-03-26T17:59:47.232559+0000","last_unstale":"2024-03-26T17:59:52.725719+0000","last_undegraded":"2024-03-26T17:59:52.725719+0000","last_fullsized":"2024-03-26T17:59:52.725719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:29:01.759592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726505+0000","last_change":"2024-03-26T17:59:47.230719+0000","last_active":"2024-03-26T17:59:52.726505+0000","last_peered":"2024-03-26T17:59:52.726505+0000","last_clean":"2024-03-26T17:59:52.726505+0000","last_became_active":"2024-03-26T17:59:47.230411+0000","last_became_peered":"2024-03-26T17:59:47.230411+0000","last_unstale":"2024-03-26T17:59:52.726505+0000","last_undegraded":"2024-03-26T17:59:52.726505+0000","last_fullsized":"2024-03-26T17:59:52.726505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:29:21.410798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726213+0000","last_change":"2024-03-26T17:59:47.233625+0000","last_active":"2024-03-26T17:59:52.726213+0000","last_peered":"2024-03-26T17:59:52.726213+0000","last_clean":"2024-03-26T17:59:52.726213+0000","last_became_active":"2024-03-26T17:59:47.233318+0000","last_became_peered":"2024-03-26T17:59:47.233318+0000","last_unstale":"2024-03-26T17:59:52.726213+0000","last_undegraded":"2024-03-26T17:59:52.726213+0000","last_fullsized":"2024-03-26T17:59:52.726213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:53:08.524508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724823+0000","last_change":"2024-03-26T17:59:47.234380+0000","last_active":"2024-03-26T17:59:52.724823+0000","last_peered":"2024-03-26T17:59:52.724823+0000","last_clean":"2024-03-26T17:59:52.724823+0000","last_became_active":"2024-03-26T17:59:47.233804+0000","last_became_peered":"2024-03-26T17:59:47.233804+0000","last_unstale":"2024-03-26T17:59:52.724823+0000","last_undegraded":"2024-03-26T17:59:52.724823+0000","last_fullsized":"2024-03-26T17:59:52.724823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:26:59.956704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461492+0000","last_change":"2024-03-26T17:59:47.232523+0000","last_active":"2024-03-26T17:59:52.461492+0000","last_peered":"2024-03-26T17:59:52.461492+0000","last_clean":"2024-03-26T17:59:52.461492+0000","last_became_active":"2024-03-26T17:59:47.232183+0000","last_became_peered":"2024-03-26T17:59:47.232183+0000","last_unstale":"2024-03-26T17:59:52.461492+0000","last_undegraded":"2024-03-26T17:59:52.461492+0000","last_fullsized":"2024-03-26T17:59:52.461492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:47:06.538092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.668953+0000","last_change":"2024-03-26T17:59:47.229078+0000","last_active":"2024-03-26T17:59:52.668953+0000","last_peered":"2024-03-26T17:59:52.668953+0000","last_clean":"2024-03-26T17:59:52.668953+0000","last_became_active":"2024-03-26T17:59:47.228757+0000","last_became_peered":"2024-03-26T17:59:47.228757+0000","last_unstale":"2024-03-26T17:59:52.668953+0000","last_undegraded":"2024-03-26T17:59:52.668953+0000","last_fullsized":"2024-03-26T17:59:52.668953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:41:32.459219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461723+0000","last_change":"2024-03-26T17:59:47.232111+0000","last_active":"2024-03-26T17:59:52.461723+0000","last_peered":"2024-03-26T17:59:52.461723+0000","last_clean":"2024-03-26T17:59:52.461723+0000","last_became_active":"2024-03-26T17:59:47.231440+0000","last_became_peered":"2024-03-26T17:59:47.231440+0000","last_unstale":"2024-03-26T17:59:52.461723+0000","last_undegraded":"2024-03-26T17:59:52.461723+0000","last_fullsized":"2024-03-26T17:59:52.461723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:23:11.607084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724386+0000","last_change":"2024-03-26T17:59:47.231875+0000","last_active":"2024-03-26T17:59:52.724386+0000","last_peered":"2024-03-26T17:59:52.724386+0000","last_clean":"2024-03-26T17:59:52.724386+0000","last_became_active":"2024-03-26T17:59:47.231534+0000","last_became_peered":"2024-03-26T17:59:47.231534+0000","last_unstale":"2024-03-26T17:59:52.724386+0000","last_undegraded":"2024-03-26T17:59:52.724386+0000","last_fullsized":"2024-03-26T17:59:52.724386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:05:33.010733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724633+0000","last_change":"2024-03-26T17:59:47.235321+0000","last_active":"2024-03-26T17:59:52.724633+0000","last_peered":"2024-03-26T17:59:52.724633+0000","last_clean":"2024-03-26T17:59:52.724633+0000","last_became_active":"2024-03-26T17:59:47.231778+0000","last_became_peered":"2024-03-26T17:59:47.231778+0000","last_unstale":"2024-03-26T17:59:52.724633+0000","last_undegraded":"2024-03-26T17:59:52.724633+0000","last_fullsized":"2024-03-26T17:59:52.724633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:58:25.613249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.669258+0000","last_change":"2024-03-26T17:59:47.224272+0000","last_active":"2024-03-26T17:59:52.669258+0000","last_peered":"2024-03-26T17:59:52.669258+0000","last_clean":"2024-03-26T17:59:52.669258+0000","last_became_active":"2024-03-26T17:59:47.223439+0000","last_became_peered":"2024-03-26T17:59:47.223439+0000","last_unstale":"2024-03-26T17:59:52.669258+0000","last_undegraded":"2024-03-26T17:59:52.669258+0000","last_fullsized":"2024-03-26T17:59:52.669258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:28:40.428780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462312+0000","last_change":"2024-03-26T17:59:47.227589+0000","last_active":"2024-03-26T17:59:52.462312+0000","last_peered":"2024-03-26T17:59:52.462312+0000","last_clean":"2024-03-26T17:59:52.462312+0000","last_became_active":"2024-03-26T17:59:47.227134+0000","last_became_peered":"2024-03-26T17:59:47.227134+0000","last_unstale":"2024-03-26T17:59:52.462312+0000","last_undegraded":"2024-03-26T17:59:52.462312+0000","last_fullsized":"2024-03-26T17:59:52.462312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:39:33.093899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724741+0000","last_change":"2024-03-26T17:59:47.233329+0000","last_active":"2024-03-26T17:59:52.724741+0000","last_peered":"2024-03-26T17:59:52.724741+0000","last_clean":"2024-03-26T17:59:52.724741+0000","last_became_active":"2024-03-26T17:59:47.233039+0000","last_became_peered":"2024-03-26T17:59:47.233039+0000","last_unstale":"2024-03-26T17:59:52.724741+0000","last_undegraded":"2024-03-26T17:59:52.724741+0000","last_fullsized":"2024-03-26T17:59:52.724741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:35:43.280834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461993+0000","last_change":"2024-03-26T17:59:47.234468+0000","last_active":"2024-03-26T17:59:52.461993+0000","last_peered":"2024-03-26T17:59:52.461993+0000","last_clean":"2024-03-26T17:59:52.461993+0000","last_became_active":"2024-03-26T17:59:47.234063+0000","last_became_peered":"2024-03-26T17:59:47.234063+0000","last_unstale":"2024-03-26T17:59:52.461993+0000","last_undegraded":"2024-03-26T17:59:52.461993+0000","last_fullsized":"2024-03-26T17:59:52.461993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:25.542893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724869+0000","last_change":"2024-03-26T17:59:47.229076+0000","last_active":"2024-03-26T17:59:52.724869+0000","last_peered":"2024-03-26T17:59:52.724869+0000","last_clean":"2024-03-26T17:59:52.724869+0000","last_became_active":"2024-03-26T17:59:47.228719+0000","last_became_peered":"2024-03-26T17:59:47.228719+0000","last_unstale":"2024-03-26T17:59:52.724869+0000","last_undegraded":"2024-03-26T17:59:52.724869+0000","last_fullsized":"2024-03-26T17:59:52.724869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:03:15.085812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724835+0000","last_change":"2024-03-26T17:59:47.239166+0000","last_active":"2024-03-26T17:59:52.724835+0000","last_peered":"2024-03-26T17:59:52.724835+0000","last_clean":"2024-03-26T17:59:52.724835+0000","last_became_active":"2024-03-26T17:59:47.236051+0000","last_became_peered":"2024-03-26T17:59:47.236051+0000","last_unstale":"2024-03-26T17:59:52.724835+0000","last_undegraded":"2024-03-26T17:59:52.724835+0000","last_fullsized":"2024-03-26T17:59:52.724835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:46:36.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.723905+0000","last_change":"2024-03-26T17:59:47.231246+0000","last_active":"2024-03-26T17:59:52.723905+0000","last_peered":"2024-03-26T17:59:52.723905+0000","last_clean":"2024-03-26T17:59:52.723905+0000","last_became_active":"2024-03-26T17:59:47.228587+0000","last_became_peered":"2024-03-26T17:59:47.228587+0000","last_unstale":"2024-03-26T17:59:52.723905+0000","last_undegraded":"2024-03-26T17:59:52.723905+0000","last_fullsized":"2024-03-26T17:59:52.723905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:02:25.850361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725614+0000","last_change":"2024-03-26T17:59:47.234790+0000","last_active":"2024-03-26T17:59:52.725614+0000","last_peered":"2024-03-26T17:59:52.725614+0000","last_clean":"2024-03-26T17:59:52.725614+0000","last_became_active":"2024-03-26T17:59:47.234436+0000","last_became_peered":"2024-03-26T17:59:47.234436+0000","last_unstale":"2024-03-26T17:59:52.725614+0000","last_undegraded":"2024-03-26T17:59:52.725614+0000","last_fullsized":"2024-03-26T17:59:52.725614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:54:26.949089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725352+0000","last_change":"2024-03-26T17:59:47.238064+0000","last_active":"2024-03-26T17:59:52.725352+0000","last_peered":"2024-03-26T17:59:52.725352+0000","last_clean":"2024-03-26T17:59:52.725352+0000","last_became_active":"2024-03-26T17:59:47.237754+0000","last_became_peered":"2024-03-26T17:59:47.237754+0000","last_unstale":"2024-03-26T17:59:52.725352+0000","last_undegraded":"2024-03-26T17:59:52.725352+0000","last_fullsized":"2024-03-26T17:59:52.725352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:27:29.603516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725383+0000","last_change":"2024-03-26T17:59:47.231000+0000","last_active":"2024-03-26T17:59:52.725383+0000","last_peered":"2024-03-26T17:59:52.725383+0000","last_clean":"2024-03-26T17:59:52.725383+0000","last_became_active":"2024-03-26T17:59:47.230675+0000","last_became_peered":"2024-03-26T17:59:47.230675+0000","last_unstale":"2024-03-26T17:59:52.725383+0000","last_undegraded":"2024-03-26T17:59:52.725383+0000","last_fullsized":"2024-03-26T17:59:52.725383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:56:59.839962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725877+0000","last_change":"2024-03-26T17:59:51.480711+0000","last_active":"2024-03-26T17:59:52.725877+0000","last_peered":"2024-03-26T17:59:52.725877+0000","last_clean":"2024-03-26T17:59:52.725877+0000","last_became_active":"2024-03-26T17:59:51.480111+0000","last_became_peered":"2024-03-26T17:59:51.480111+0000","last_unstale":"2024-03-26T17:59:52.725877+0000","last_undegraded":"2024-03-26T17:59:52.725877+0000","last_fullsized":"2024-03-26T17:59:52.725877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:40:55.456138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725787+0000","last_change":"2024-03-26T17:59:46.434642+0000","last_active":"2024-03-26T17:59:52.725787+0000","last_peered":"2024-03-26T17:59:52.725787+0000","last_clean":"2024-03-26T17:59:52.725787+0000","last_became_active":"2024-03-26T17:59:46.434209+0000","last_became_peered":"2024-03-26T17:59:46.434209+0000","last_unstale":"2024-03-26T17:59:52.725787+0000","last_undegraded":"2024-03-26T17:59:52.725787+0000","last_fullsized":"2024-03-26T17:59:52.725787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:55:05.365712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725909+0000","last_change":"2024-03-26T17:59:47.239356+0000","last_active":"2024-03-26T17:59:52.725909+0000","last_peered":"2024-03-26T17:59:52.725909+0000","last_clean":"2024-03-26T17:59:52.725909+0000","last_became_active":"2024-03-26T17:59:47.239020+0000","last_became_peered":"2024-03-26T17:59:47.239020+0000","last_unstale":"2024-03-26T17:59:52.725909+0000","last_undegraded":"2024-03-26T17:59:52.725909+0000","last_fullsized":"2024-03-26T17:59:52.725909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:56.992379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724009+0000","last_change":"2024-03-26T17:59:51.486894+0000","last_active":"2024-03-26T17:59:52.724009+0000","last_peered":"2024-03-26T17:59:52.724009+0000","last_clean":"2024-03-26T17:59:52.724009+0000","last_became_active":"2024-03-26T17:59:51.486465+0000","last_became_peered":"2024-03-26T17:59:51.486465+0000","last_unstale":"2024-03-26T17:59:52.724009+0000","last_undegraded":"2024-03-26T17:59:52.724009+0000","last_fullsized":"2024-03-26T17:59:52.724009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:15:10.145415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726190+0000","last_change":"2024-03-26T17:59:46.437908+0000","last_active":"2024-03-26T17:59:52.726190+0000","last_peered":"2024-03-26T17:59:52.726190+0000","last_clean":"2024-03-26T17:59:52.726190+0000","last_became_active":"2024-03-26T17:59:46.437592+0000","last_became_peered":"2024-03-26T17:59:46.437592+0000","last_unstale":"2024-03-26T17:59:52.726190+0000","last_undegraded":"2024-03-26T17:59:52.726190+0000","last_fullsized":"2024-03-26T17:59:52.726190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:05.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726428+0000","last_change":"2024-03-26T17:59:47.240291+0000","last_active":"2024-03-26T17:59:52.726428+0000","last_peered":"2024-03-26T17:59:52.726428+0000","last_clean":"2024-03-26T17:59:52.726428+0000","last_became_active":"2024-03-26T17:59:47.239893+0000","last_became_peered":"2024-03-26T17:59:47.239893+0000","last_unstale":"2024-03-26T17:59:52.726428+0000","last_undegraded":"2024-03-26T17:59:52.726428+0000","last_fullsized":"2024-03-26T17:59:52.726428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:44:22.946199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724179+0000","last_change":"2024-03-26T17:59:51.483970+0000","last_active":"2024-03-26T17:59:52.724179+0000","last_peered":"2024-03-26T17:59:52.724179+0000","last_clean":"2024-03-26T17:59:52.724179+0000","last_became_active":"2024-03-26T17:59:51.483585+0000","last_became_peered":"2024-03-26T17:59:51.483585+0000","last_unstale":"2024-03-26T17:59:52.724179+0000","last_undegraded":"2024-03-26T17:59:52.724179+0000","last_fullsized":"2024-03-26T17:59:52.724179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:22:30.873421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.519270+0000","last_change":"2024-03-26T17:59:46.439374+0000","last_active":"2024-03-26T18:00:06.519270+0000","last_peered":"2024-03-26T18:00:06.519270+0000","last_clean":"2024-03-26T18:00:06.519270+0000","last_became_active":"2024-03-26T17:59:46.439065+0000","last_became_peered":"2024-03-26T17:59:46.439065+0000","last_unstale":"2024-03-26T18:00:06.519270+0000","last_undegraded":"2024-03-26T18:00:06.519270+0000","last_fullsized":"2024-03-26T18:00:06.519270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:45:37.954375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724501+0000","last_change":"2024-03-26T17:59:47.237168+0000","last_active":"2024-03-26T17:59:52.724501+0000","last_peered":"2024-03-26T17:59:52.724501+0000","last_clean":"2024-03-26T17:59:52.724501+0000","last_became_active":"2024-03-26T17:59:47.232573+0000","last_became_peered":"2024-03-26T17:59:47.232573+0000","last_unstale":"2024-03-26T17:59:52.724501+0000","last_undegraded":"2024-03-26T17:59:52.724501+0000","last_fullsized":"2024-03-26T17:59:52.724501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:00:30.236249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724004+0000","last_change":"2024-03-26T17:59:51.477500+0000","last_active":"2024-03-26T17:59:52.724004+0000","last_peered":"2024-03-26T17:59:52.724004+0000","last_clean":"2024-03-26T17:59:52.724004+0000","last_became_active":"2024-03-26T17:59:51.477064+0000","last_became_peered":"2024-03-26T17:59:51.477064+0000","last_unstale":"2024-03-26T17:59:52.724004+0000","last_undegraded":"2024-03-26T17:59:52.724004+0000","last_fullsized":"2024-03-26T17:59:52.724004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:07:05.265910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461791+0000","last_change":"2024-03-26T17:59:46.437738+0000","last_active":"2024-03-26T17:59:52.461791+0000","last_peered":"2024-03-26T17:59:52.461791+0000","last_clean":"2024-03-26T17:59:52.461791+0000","last_became_active":"2024-03-26T17:59:46.437373+0000","last_became_peered":"2024-03-26T17:59:46.437373+0000","last_unstale":"2024-03-26T17:59:52.461791+0000","last_undegraded":"2024-03-26T17:59:52.461791+0000","last_fullsized":"2024-03-26T17:59:52.461791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:49:32.700437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726060+0000","last_change":"2024-03-26T17:59:47.238169+0000","last_active":"2024-03-26T17:59:52.726060+0000","last_peered":"2024-03-26T17:59:52.726060+0000","last_clean":"2024-03-26T17:59:52.726060+0000","last_became_active":"2024-03-26T17:59:47.237854+0000","last_became_peered":"2024-03-26T17:59:47.237854+0000","last_unstale":"2024-03-26T17:59:52.726060+0000","last_undegraded":"2024-03-26T17:59:52.726060+0000","last_fullsized":"2024-03-26T17:59:52.726060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:23:09.040051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725185+0000","last_change":"2024-03-26T17:59:51.479332+0000","last_active":"2024-03-26T17:59:52.725185+0000","last_peered":"2024-03-26T17:59:52.725185+0000","last_clean":"2024-03-26T17:59:52.725185+0000","last_became_active":"2024-03-26T17:59:51.478857+0000","last_became_peered":"2024-03-26T17:59:51.478857+0000","last_unstale":"2024-03-26T17:59:52.725185+0000","last_undegraded":"2024-03-26T17:59:52.725185+0000","last_fullsized":"2024-03-26T17:59:52.725185+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:04:30.568741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725566+0000","last_change":"2024-03-26T17:59:46.438017+0000","last_active":"2024-03-26T17:59:52.725566+0000","last_peered":"2024-03-26T17:59:52.725566+0000","last_clean":"2024-03-26T17:59:52.725566+0000","last_became_active":"2024-03-26T17:59:46.437625+0000","last_became_peered":"2024-03-26T17:59:46.437625+0000","last_unstale":"2024-03-26T17:59:52.725566+0000","last_undegraded":"2024-03-26T17:59:52.725566+0000","last_fullsized":"2024-03-26T17:59:52.725566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:33:27.460626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461982+0000","last_change":"2024-03-26T17:59:47.232764+0000","last_active":"2024-03-26T17:59:52.461982+0000","last_peered":"2024-03-26T17:59:52.461982+0000","last_clean":"2024-03-26T17:59:52.461982+0000","last_became_active":"2024-03-26T17:59:47.231672+0000","last_became_peered":"2024-03-26T17:59:47.231672+0000","last_unstale":"2024-03-26T17:59:52.461982+0000","last_undegraded":"2024-03-26T17:59:52.461982+0000","last_fullsized":"2024-03-26T17:59:52.461982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.812245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725397+0000","last_change":"2024-03-26T17:59:51.476645+0000","last_active":"2024-03-26T17:59:52.725397+0000","last_peered":"2024-03-26T17:59:52.725397+0000","last_clean":"2024-03-26T17:59:52.725397+0000","last_became_active":"2024-03-26T17:59:51.476232+0000","last_became_peered":"2024-03-26T17:59:51.476232+0000","last_unstale":"2024-03-26T17:59:52.725397+0000","last_undegraded":"2024-03-26T17:59:52.725397+0000","last_fullsized":"2024-03-26T17:59:52.725397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:27:16.109125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.668891+0000","last_change":"2024-03-26T17:59:46.438232+0000","last_active":"2024-03-26T17:59:52.668891+0000","last_peered":"2024-03-26T17:59:52.668891+0000","last_clean":"2024-03-26T17:59:52.668891+0000","last_became_active":"2024-03-26T17:59:46.437871+0000","last_became_peered":"2024-03-26T17:59:46.437871+0000","last_unstale":"2024-03-26T17:59:52.668891+0000","last_undegraded":"2024-03-26T17:59:52.668891+0000","last_fullsized":"2024-03-26T17:59:52.668891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:08.626264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725969+0000","last_change":"2024-03-26T17:59:47.237144+0000","last_active":"2024-03-26T17:59:52.725969+0000","last_peered":"2024-03-26T17:59:52.725969+0000","last_clean":"2024-03-26T17:59:52.725969+0000","last_became_active":"2024-03-26T17:59:47.236696+0000","last_became_peered":"2024-03-26T17:59:47.236696+0000","last_unstale":"2024-03-26T17:59:52.725969+0000","last_undegraded":"2024-03-26T17:59:52.725969+0000","last_fullsized":"2024-03-26T17:59:52.725969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:49:31.197898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725109+0000","last_change":"2024-03-26T17:59:51.490204+0000","last_active":"2024-03-26T17:59:52.725109+0000","last_peered":"2024-03-26T17:59:52.725109+0000","last_clean":"2024-03-26T17:59:52.725109+0000","last_became_active":"2024-03-26T17:59:51.489846+0000","last_became_peered":"2024-03-26T17:59:51.489846+0000","last_unstale":"2024-03-26T17:59:52.725109+0000","last_undegraded":"2024-03-26T17:59:52.725109+0000","last_fullsized":"2024-03-26T17:59:52.725109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:26:12.284436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.520990+0000","last_change":"2024-03-26T17:59:46.429699+0000","last_active":"2024-03-26T18:00:06.520990+0000","last_peered":"2024-03-26T18:00:06.520990+0000","last_clean":"2024-03-26T18:00:06.520990+0000","last_became_active":"2024-03-26T17:59:46.429319+0000","last_became_peered":"2024-03-26T17:59:46.429319+0000","last_unstale":"2024-03-26T18:00:06.520990+0000","last_undegraded":"2024-03-26T18:00:06.520990+0000","last_fullsized":"2024-03-26T18:00:06.520990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:05:15.695696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724666+0000","last_change":"2024-03-26T17:59:47.239705+0000","last_active":"2024-03-26T17:59:52.724666+0000","last_peered":"2024-03-26T17:59:52.724666+0000","last_clean":"2024-03-26T17:59:52.724666+0000","last_became_active":"2024-03-26T17:59:47.239046+0000","last_became_peered":"2024-03-26T17:59:47.239046+0000","last_unstale":"2024-03-26T17:59:52.724666+0000","last_undegraded":"2024-03-26T17:59:52.724666+0000","last_fullsized":"2024-03-26T17:59:52.724666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:08:34.930460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726435+0000","last_change":"2024-03-26T17:59:51.477511+0000","last_active":"2024-03-26T17:59:52.726435+0000","last_peered":"2024-03-26T17:59:52.726435+0000","last_clean":"2024-03-26T17:59:52.726435+0000","last_became_active":"2024-03-26T17:59:51.473949+0000","last_became_peered":"2024-03-26T17:59:51.473949+0000","last_unstale":"2024-03-26T17:59:52.726435+0000","last_undegraded":"2024-03-26T17:59:52.726435+0000","last_fullsized":"2024-03-26T17:59:52.726435+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:26:01.431977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726138+0000","last_change":"2024-03-26T17:59:46.433235+0000","last_active":"2024-03-26T17:59:52.726138+0000","last_peered":"2024-03-26T17:59:52.726138+0000","last_clean":"2024-03-26T17:59:52.726138+0000","last_became_active":"2024-03-26T17:59:46.432929+0000","last_became_peered":"2024-03-26T17:59:46.432929+0000","last_unstale":"2024-03-26T17:59:52.726138+0000","last_undegraded":"2024-03-26T17:59:52.726138+0000","last_fullsized":"2024-03-26T17:59:52.726138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:46:23.064447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724317+0000","last_change":"2024-03-26T17:59:47.232526+0000","last_active":"2024-03-26T17:59:52.724317+0000","last_peered":"2024-03-26T17:59:52.724317+0000","last_clean":"2024-03-26T17:59:52.724317+0000","last_became_active":"2024-03-26T17:59:47.232216+0000","last_became_peered":"2024-03-26T17:59:47.232216+0000","last_unstale":"2024-03-26T17:59:52.724317+0000","last_undegraded":"2024-03-26T17:59:52.724317+0000","last_fullsized":"2024-03-26T17:59:52.724317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:20:07.510565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725141+0000","last_change":"2024-03-26T17:59:51.482706+0000","last_active":"2024-03-26T17:59:52.725141+0000","last_peered":"2024-03-26T17:59:52.725141+0000","last_clean":"2024-03-26T17:59:52.725141+0000","last_became_active":"2024-03-26T17:59:51.482312+0000","last_became_peered":"2024-03-26T17:59:51.482312+0000","last_unstale":"2024-03-26T17:59:52.725141+0000","last_undegraded":"2024-03-26T17:59:52.725141+0000","last_fullsized":"2024-03-26T17:59:52.725141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:42:26.614170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.669151+0000","last_change":"2024-03-26T17:59:46.440919+0000","last_active":"2024-03-26T17:59:52.669151+0000","last_peered":"2024-03-26T17:59:52.669151+0000","last_clean":"2024-03-26T17:59:52.669151+0000","last_became_active":"2024-03-26T17:59:46.440579+0000","last_became_peered":"2024-03-26T17:59:46.440579+0000","last_unstale":"2024-03-26T17:59:52.669151+0000","last_undegraded":"2024-03-26T17:59:52.669151+0000","last_fullsized":"2024-03-26T17:59:52.669151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:38:27.325478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724495+0000","last_change":"2024-03-26T17:59:47.236251+0000","last_active":"2024-03-26T17:59:52.724495+0000","last_peered":"2024-03-26T17:59:52.724495+0000","last_clean":"2024-03-26T17:59:52.724495+0000","last_became_active":"2024-03-26T17:59:47.235941+0000","last_became_peered":"2024-03-26T17:59:47.235941+0000","last_unstale":"2024-03-26T17:59:52.724495+0000","last_undegraded":"2024-03-26T17:59:52.724495+0000","last_fullsized":"2024-03-26T17:59:52.724495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:50:34.821242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724238+0000","last_change":"2024-03-26T17:59:51.480155+0000","last_active":"2024-03-26T17:59:52.724238+0000","last_peered":"2024-03-26T17:59:52.724238+0000","last_clean":"2024-03-26T17:59:52.724238+0000","last_became_active":"2024-03-26T17:59:51.479761+0000","last_became_peered":"2024-03-26T17:59:51.479761+0000","last_unstale":"2024-03-26T17:59:52.724238+0000","last_undegraded":"2024-03-26T17:59:52.724238+0000","last_fullsized":"2024-03-26T17:59:52.724238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:03:44.256586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.510913+0000","last_change":"2024-03-26T17:59:46.436559+0000","last_active":"2024-03-26T18:00:05.510913+0000","last_peered":"2024-03-26T18:00:05.510913+0000","last_clean":"2024-03-26T18:00:05.510913+0000","last_became_active":"2024-03-26T17:59:46.436224+0000","last_became_peered":"2024-03-26T17:59:46.436224+0000","last_unstale":"2024-03-26T18:00:05.510913+0000","last_undegraded":"2024-03-26T18:00:05.510913+0000","last_fullsized":"2024-03-26T18:00:05.510913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:12:56.716278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726538+0000","last_change":"2024-03-26T17:59:47.240266+0000","last_active":"2024-03-26T17:59:52.726538+0000","last_peered":"2024-03-26T17:59:52.726538+0000","last_clean":"2024-03-26T17:59:52.726538+0000","last_became_active":"2024-03-26T17:59:47.238847+0000","last_became_peered":"2024-03-26T17:59:47.238847+0000","last_unstale":"2024-03-26T17:59:52.726538+0000","last_undegraded":"2024-03-26T17:59:52.726538+0000","last_fullsized":"2024-03-26T17:59:52.726538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:47:36.325585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724840+0000","last_change":"2024-03-26T17:59:51.487207+0000","last_active":"2024-03-26T17:59:52.724840+0000","last_peered":"2024-03-26T17:59:52.724840+0000","last_clean":"2024-03-26T17:59:52.724840+0000","last_became_active":"2024-03-26T17:59:51.486351+0000","last_became_peered":"2024-03-26T17:59:51.486351+0000","last_unstale":"2024-03-26T17:59:52.724840+0000","last_undegraded":"2024-03-26T17:59:52.724840+0000","last_fullsized":"2024-03-26T17:59:52.724840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:35:03.445677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725874+0000","last_change":"2024-03-26T17:59:46.437839+0000","last_active":"2024-03-26T17:59:52.725874+0000","last_peered":"2024-03-26T17:59:52.725874+0000","last_clean":"2024-03-26T17:59:52.725874+0000","last_became_active":"2024-03-26T17:59:46.437515+0000","last_became_peered":"2024-03-26T17:59:46.437515+0000","last_unstale":"2024-03-26T17:59:52.725874+0000","last_undegraded":"2024-03-26T17:59:52.725874+0000","last_fullsized":"2024-03-26T17:59:52.725874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:38:34.875841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724596+0000","last_change":"2024-03-26T17:59:47.234833+0000","last_active":"2024-03-26T17:59:52.724596+0000","last_peered":"2024-03-26T17:59:52.724596+0000","last_clean":"2024-03-26T17:59:52.724596+0000","last_became_active":"2024-03-26T17:59:47.232925+0000","last_became_peered":"2024-03-26T17:59:47.232925+0000","last_unstale":"2024-03-26T17:59:52.724596+0000","last_undegraded":"2024-03-26T17:59:52.724596+0000","last_fullsized":"2024-03-26T17:59:52.724596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:42:23.618396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724699+0000","last_change":"2024-03-26T17:59:51.481550+0000","last_active":"2024-03-26T17:59:52.724699+0000","last_peered":"2024-03-26T17:59:52.724699+0000","last_clean":"2024-03-26T17:59:52.724699+0000","last_became_active":"2024-03-26T17:59:51.481142+0000","last_became_peered":"2024-03-26T17:59:51.481142+0000","last_unstale":"2024-03-26T17:59:52.724699+0000","last_undegraded":"2024-03-26T17:59:52.724699+0000","last_fullsized":"2024-03-26T17:59:52.724699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:37.529859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'10","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:03.132117+0000","last_change":"2024-03-26T17:59:46.438936+0000","last_active":"2024-03-26T18:00:03.132117+0000","last_peered":"2024-03-26T18:00:03.132117+0000","last_clean":"2024-03-26T18:00:03.132117+0000","last_became_active":"2024-03-26T17:59:46.438609+0000","last_became_peered":"2024-03-26T17:59:46.438609+0000","last_unstale":"2024-03-26T18:00:03.132117+0000","last_undegraded":"2024-03-26T18:00:03.132117+0000","last_fullsized":"2024-03-26T18:00:03.132117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:38:36.333660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461848+0000","last_change":"2024-03-26T17:59:47.224495+0000","last_active":"2024-03-26T17:59:52.461848+0000","last_peered":"2024-03-26T17:59:52.461848+0000","last_clean":"2024-03-26T17:59:52.461848+0000","last_became_active":"2024-03-26T17:59:47.224184+0000","last_became_peered":"2024-03-26T17:59:47.224184+0000","last_unstale":"2024-03-26T17:59:52.461848+0000","last_undegraded":"2024-03-26T17:59:52.461848+0000","last_fullsized":"2024-03-26T17:59:52.461848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:17:23.028748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726584+0000","last_change":"2024-03-26T17:59:51.483006+0000","last_active":"2024-03-26T17:59:52.726584+0000","last_peered":"2024-03-26T17:59:52.726584+0000","last_clean":"2024-03-26T17:59:52.726584+0000","last_became_active":"2024-03-26T17:59:51.482281+0000","last_became_peered":"2024-03-26T17:59:51.482281+0000","last_unstale":"2024-03-26T17:59:52.726584+0000","last_undegraded":"2024-03-26T17:59:52.726584+0000","last_fullsized":"2024-03-26T17:59:52.726584+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:05:25.322809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724907+0000","last_change":"2024-03-26T17:59:46.440480+0000","last_active":"2024-03-26T17:59:52.724907+0000","last_peered":"2024-03-26T17:59:52.724907+0000","last_clean":"2024-03-26T17:59:52.724907+0000","last_became_active":"2024-03-26T17:59:46.439574+0000","last_became_peered":"2024-03-26T17:59:46.439574+0000","last_unstale":"2024-03-26T17:59:52.724907+0000","last_undegraded":"2024-03-26T17:59:52.724907+0000","last_fullsized":"2024-03-26T17:59:52.724907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:50:42.296440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461473+0000","last_change":"2024-03-26T17:59:47.229944+0000","last_active":"2024-03-26T17:59:52.461473+0000","last_peered":"2024-03-26T17:59:52.461473+0000","last_clean":"2024-03-26T17:59:52.461473+0000","last_became_active":"2024-03-26T17:59:47.229626+0000","last_became_peered":"2024-03-26T17:59:47.229626+0000","last_unstale":"2024-03-26T17:59:52.461473+0000","last_undegraded":"2024-03-26T17:59:52.461473+0000","last_fullsized":"2024-03-26T17:59:52.461473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:18.998051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724804+0000","last_change":"2024-03-26T17:59:51.486939+0000","last_active":"2024-03-26T17:59:52.724804+0000","last_peered":"2024-03-26T17:59:52.724804+0000","last_clean":"2024-03-26T17:59:52.724804+0000","last_became_active":"2024-03-26T17:59:51.486458+0000","last_became_peered":"2024-03-26T17:59:51.486458+0000","last_unstale":"2024-03-26T17:59:52.724804+0000","last_undegraded":"2024-03-26T17:59:52.724804+0000","last_fullsized":"2024-03-26T17:59:52.724804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:30:25.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725825+0000","last_change":"2024-03-26T17:59:46.435999+0000","last_active":"2024-03-26T17:59:52.725825+0000","last_peered":"2024-03-26T17:59:52.725825+0000","last_clean":"2024-03-26T17:59:52.725825+0000","last_became_active":"2024-03-26T17:59:46.435611+0000","last_became_peered":"2024-03-26T17:59:46.435611+0000","last_unstale":"2024-03-26T17:59:52.725825+0000","last_undegraded":"2024-03-26T17:59:52.725825+0000","last_fullsized":"2024-03-26T17:59:52.725825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:13:14.904792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462006+0000","last_change":"2024-03-26T17:59:47.236395+0000","last_active":"2024-03-26T17:59:52.462006+0000","last_peered":"2024-03-26T17:59:52.462006+0000","last_clean":"2024-03-26T17:59:52.462006+0000","last_became_active":"2024-03-26T17:59:47.236055+0000","last_became_peered":"2024-03-26T17:59:47.236055+0000","last_unstale":"2024-03-26T17:59:52.462006+0000","last_undegraded":"2024-03-26T17:59:52.462006+0000","last_fullsized":"2024-03-26T17:59:52.462006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:06:26.131419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725252+0000","last_change":"2024-03-26T17:59:51.484584+0000","last_active":"2024-03-26T17:59:52.725252+0000","last_peered":"2024-03-26T17:59:52.725252+0000","last_clean":"2024-03-26T17:59:52.725252+0000","last_became_active":"2024-03-26T17:59:51.484138+0000","last_became_peered":"2024-03-26T17:59:51.484138+0000","last_unstale":"2024-03-26T17:59:52.725252+0000","last_undegraded":"2024-03-26T17:59:52.725252+0000","last_fullsized":"2024-03-26T17:59:52.725252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:49:40.163174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.518214+0000","last_change":"2024-03-26T17:59:46.433243+0000","last_active":"2024-03-26T18:00:06.518214+0000","last_peered":"2024-03-26T18:00:06.518214+0000","last_clean":"2024-03-26T18:00:06.518214+0000","last_became_active":"2024-03-26T17:59:46.432939+0000","last_became_peered":"2024-03-26T17:59:46.432939+0000","last_unstale":"2024-03-26T18:00:06.518214+0000","last_undegraded":"2024-03-26T18:00:06.518214+0000","last_fullsized":"2024-03-26T18:00:06.518214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:44:09.473862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461770+0000","last_change":"2024-03-26T17:59:47.228882+0000","last_active":"2024-03-26T17:59:52.461770+0000","last_peered":"2024-03-26T17:59:52.461770+0000","last_clean":"2024-03-26T17:59:52.461770+0000","last_became_active":"2024-03-26T17:59:47.228601+0000","last_became_peered":"2024-03-26T17:59:47.228601+0000","last_unstale":"2024-03-26T17:59:52.461770+0000","last_undegraded":"2024-03-26T17:59:52.461770+0000","last_fullsized":"2024-03-26T17:59:52.461770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:07:46.835582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462024+0000","last_change":"2024-03-26T17:59:51.488010+0000","last_active":"2024-03-26T17:59:52.462024+0000","last_peered":"2024-03-26T17:59:52.462024+0000","last_clean":"2024-03-26T17:59:52.462024+0000","last_became_active":"2024-03-26T17:59:51.486406+0000","last_became_peered":"2024-03-26T17:59:51.486406+0000","last_unstale":"2024-03-26T17:59:52.462024+0000","last_undegraded":"2024-03-26T17:59:52.462024+0000","last_fullsized":"2024-03-26T17:59:52.462024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:54:49.102720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.521363+0000","last_change":"2024-03-26T17:59:46.438764+0000","last_active":"2024-03-26T18:00:06.521363+0000","last_peered":"2024-03-26T18:00:06.521363+0000","last_clean":"2024-03-26T18:00:06.521363+0000","last_became_active":"2024-03-26T17:59:46.438329+0000","last_became_peered":"2024-03-26T17:59:46.438329+0000","last_unstale":"2024-03-26T18:00:06.521363+0000","last_undegraded":"2024-03-26T18:00:06.521363+0000","last_fullsized":"2024-03-26T18:00:06.521363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:33:19.025134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725776+0000","last_change":"2024-03-26T17:59:47.237020+0000","last_active":"2024-03-26T17:59:52.725776+0000","last_peered":"2024-03-26T17:59:52.725776+0000","last_clean":"2024-03-26T17:59:52.725776+0000","last_became_active":"2024-03-26T17:59:47.236693+0000","last_became_peered":"2024-03-26T17:59:47.236693+0000","last_unstale":"2024-03-26T17:59:52.725776+0000","last_undegraded":"2024-03-26T17:59:52.725776+0000","last_fullsized":"2024-03-26T17:59:52.725776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:40:25.473994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725693+0000","last_change":"2024-03-26T17:59:51.482799+0000","last_active":"2024-03-26T17:59:52.725693+0000","last_peered":"2024-03-26T17:59:52.725693+0000","last_clean":"2024-03-26T17:59:52.725693+0000","last_became_active":"2024-03-26T17:59:51.482246+0000","last_became_peered":"2024-03-26T17:59:51.482246+0000","last_unstale":"2024-03-26T17:59:52.725693+0000","last_undegraded":"2024-03-26T17:59:52.725693+0000","last_fullsized":"2024-03-26T17:59:52.725693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:43:11.132952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461730+0000","last_change":"2024-03-26T17:59:46.438038+0000","last_active":"2024-03-26T17:59:52.461730+0000","last_peered":"2024-03-26T17:59:52.461730+0000","last_clean":"2024-03-26T17:59:52.461730+0000","last_became_active":"2024-03-26T17:59:46.437654+0000","last_became_peered":"2024-03-26T17:59:46.437654+0000","last_unstale":"2024-03-26T17:59:52.461730+0000","last_undegraded":"2024-03-26T17:59:52.461730+0000","last_fullsized":"2024-03-26T17:59:52.461730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:33:04.341028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724919+0000","last_change":"2024-03-26T17:59:47.235516+0000","last_active":"2024-03-26T17:59:52.724919+0000","last_peered":"2024-03-26T17:59:52.724919+0000","last_clean":"2024-03-26T17:59:52.724919+0000","last_became_active":"2024-03-26T17:59:47.235182+0000","last_became_peered":"2024-03-26T17:59:47.235182+0000","last_unstale":"2024-03-26T17:59:52.724919+0000","last_undegraded":"2024-03-26T17:59:52.724919+0000","last_fullsized":"2024-03-26T17:59:52.724919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:28:53.365700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461805+0000","last_change":"2024-03-26T17:59:51.477057+0000","last_active":"2024-03-26T17:59:52.461805+0000","last_peered":"2024-03-26T17:59:52.461805+0000","last_clean":"2024-03-26T17:59:52.461805+0000","last_became_active":"2024-03-26T17:59:51.476576+0000","last_became_peered":"2024-03-26T17:59:51.476576+0000","last_unstale":"2024-03-26T17:59:52.461805+0000","last_undegraded":"2024-03-26T17:59:52.461805+0000","last_fullsized":"2024-03-26T17:59:52.461805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:42.533519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725178+0000","last_change":"2024-03-26T17:59:46.437494+0000","last_active":"2024-03-26T17:59:52.725178+0000","last_peered":"2024-03-26T17:59:52.725178+0000","last_clean":"2024-03-26T17:59:52.725178+0000","last_became_active":"2024-03-26T17:59:46.437184+0000","last_became_peered":"2024-03-26T17:59:46.437184+0000","last_unstale":"2024-03-26T17:59:52.725178+0000","last_undegraded":"2024-03-26T17:59:52.725178+0000","last_fullsized":"2024-03-26T17:59:52.725178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:28:21.560104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.669230+0000","last_change":"2024-03-26T17:59:47.232436+0000","last_active":"2024-03-26T17:59:52.669230+0000","last_peered":"2024-03-26T17:59:52.669230+0000","last_clean":"2024-03-26T17:59:52.669230+0000","last_became_active":"2024-03-26T17:59:47.232129+0000","last_became_peered":"2024-03-26T17:59:47.232129+0000","last_unstale":"2024-03-26T17:59:52.669230+0000","last_undegraded":"2024-03-26T17:59:52.669230+0000","last_fullsized":"2024-03-26T17:59:52.669230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:27:54.504624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461845+0000","last_change":"2024-03-26T17:59:51.475972+0000","last_active":"2024-03-26T17:59:52.461845+0000","last_peered":"2024-03-26T17:59:52.461845+0000","last_clean":"2024-03-26T17:59:52.461845+0000","last_became_active":"2024-03-26T17:59:51.475531+0000","last_became_peered":"2024-03-26T17:59:51.475531+0000","last_unstale":"2024-03-26T17:59:52.461845+0000","last_undegraded":"2024-03-26T17:59:52.461845+0000","last_fullsized":"2024-03-26T17:59:52.461845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:43:23.639809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461902+0000","last_change":"2024-03-26T17:59:46.432980+0000","last_active":"2024-03-26T17:59:52.461902+0000","last_peered":"2024-03-26T17:59:52.461902+0000","last_clean":"2024-03-26T17:59:52.461902+0000","last_became_active":"2024-03-26T17:59:46.432095+0000","last_became_peered":"2024-03-26T17:59:46.432095+0000","last_unstale":"2024-03-26T17:59:52.461902+0000","last_undegraded":"2024-03-26T17:59:52.461902+0000","last_fullsized":"2024-03-26T17:59:52.461902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:19:36.897555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.669072+0000","last_change":"2024-03-26T17:59:47.229623+0000","last_active":"2024-03-26T17:59:52.669072+0000","last_peered":"2024-03-26T17:59:52.669072+0000","last_clean":"2024-03-26T17:59:52.669072+0000","last_became_active":"2024-03-26T17:59:47.228781+0000","last_became_peered":"2024-03-26T17:59:47.228781+0000","last_unstale":"2024-03-26T17:59:52.669072+0000","last_undegraded":"2024-03-26T17:59:52.669072+0000","last_fullsized":"2024-03-26T17:59:52.669072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:54:22.171273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461999+0000","last_change":"2024-03-26T17:59:51.483247+0000","last_active":"2024-03-26T17:59:52.461999+0000","last_peered":"2024-03-26T17:59:52.461999+0000","last_clean":"2024-03-26T17:59:52.461999+0000","last_became_active":"2024-03-26T17:59:51.482854+0000","last_became_peered":"2024-03-26T17:59:51.482854+0000","last_unstale":"2024-03-26T17:59:52.461999+0000","last_undegraded":"2024-03-26T17:59:52.461999+0000","last_fullsized":"2024-03-26T17:59:52.461999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:24:13.532986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726549+0000","last_change":"2024-03-26T17:59:47.238866+0000","last_active":"2024-03-26T17:59:52.726549+0000","last_peered":"2024-03-26T17:59:52.726549+0000","last_clean":"2024-03-26T17:59:52.726549+0000","last_became_active":"2024-03-26T17:59:47.238366+0000","last_became_peered":"2024-03-26T17:59:47.238366+0000","last_unstale":"2024-03-26T17:59:52.726549+0000","last_undegraded":"2024-03-26T17:59:52.726549+0000","last_fullsized":"2024-03-26T17:59:52.726549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:04.045820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.522403+0000","last_change":"2024-03-26T17:59:46.433046+0000","last_active":"2024-03-26T18:00:06.522403+0000","last_peered":"2024-03-26T18:00:06.522403+0000","last_clean":"2024-03-26T18:00:06.522403+0000","last_became_active":"2024-03-26T17:59:46.432585+0000","last_became_peered":"2024-03-26T17:59:46.432585+0000","last_unstale":"2024-03-26T18:00:06.522403+0000","last_undegraded":"2024-03-26T18:00:06.522403+0000","last_fullsized":"2024-03-26T18:00:06.522403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:17.051521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462236+0000","last_change":"2024-03-26T17:59:51.478866+0000","last_active":"2024-03-26T17:59:52.462236+0000","last_peered":"2024-03-26T17:59:52.462236+0000","last_clean":"2024-03-26T17:59:52.462236+0000","last_became_active":"2024-03-26T17:59:51.478497+0000","last_became_peered":"2024-03-26T17:59:51.478497+0000","last_unstale":"2024-03-26T17:59:52.462236+0000","last_undegraded":"2024-03-26T17:59:52.462236+0000","last_fullsized":"2024-03-26T17:59:52.462236+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:15:05.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725181+0000","last_change":"2024-03-26T17:59:47.232349+0000","last_active":"2024-03-26T17:59:52.725181+0000","last_peered":"2024-03-26T17:59:52.725181+0000","last_clean":"2024-03-26T17:59:52.725181+0000","last_became_active":"2024-03-26T17:59:47.232001+0000","last_became_peered":"2024-03-26T17:59:47.232001+0000","last_unstale":"2024-03-26T17:59:52.725181+0000","last_undegraded":"2024-03-26T17:59:52.725181+0000","last_fullsized":"2024-03-26T17:59:52.725181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:42:14.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.513016+0000","last_change":"2024-03-26T17:59:46.431522+0000","last_active":"2024-03-26T18:00:05.513016+0000","last_peered":"2024-03-26T18:00:05.513016+0000","last_clean":"2024-03-26T18:00:05.513016+0000","last_became_active":"2024-03-26T17:59:46.431201+0000","last_became_peered":"2024-03-26T17:59:46.431201+0000","last_unstale":"2024-03-26T18:00:05.513016+0000","last_undegraded":"2024-03-26T18:00:05.513016+0000","last_fullsized":"2024-03-26T18:00:05.513016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:11.298111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461649+0000","last_change":"2024-03-26T17:58:38.167621+0000","last_active":"2024-03-26T17:59:52.461649+0000","last_peered":"2024-03-26T17:59:52.461649+0000","last_clean":"2024-03-26T17:59:52.461649+0000","last_became_active":"2024-03-26T17:58:38.167247+0000","last_became_peered":"2024-03-26T17:58:38.167247+0000","last_unstale":"2024-03-26T17:59:52.461649+0000","last_undegraded":"2024-03-26T17:59:52.461649+0000","last_fullsized":"2024-03-26T17:59:52.461649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:55:11.216723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:55:11.216723+0000","last_clean_scrub_stamp":"2024-03-26T17:55:11.216723+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:59:04.140225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726085+0000","last_change":"2024-03-26T17:59:51.480354+0000","last_active":"2024-03-26T17:59:52.726085+0000","last_peered":"2024-03-26T17:59:52.726085+0000","last_clean":"2024-03-26T17:59:52.726085+0000","last_became_active":"2024-03-26T17:59:51.479474+0000","last_became_peered":"2024-03-26T17:59:51.479474+0000","last_unstale":"2024-03-26T17:59:52.726085+0000","last_undegraded":"2024-03-26T17:59:52.726085+0000","last_fullsized":"2024-03-26T17:59:52.726085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:48:31.846364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461227+0000","last_change":"2024-03-26T17:59:47.226036+0000","last_active":"2024-03-26T17:59:52.461227+0000","last_peered":"2024-03-26T17:59:52.461227+0000","last_clean":"2024-03-26T17:59:52.461227+0000","last_became_active":"2024-03-26T17:59:47.225722+0000","last_became_peered":"2024-03-26T17:59:47.225722+0000","last_unstale":"2024-03-26T17:59:52.461227+0000","last_undegraded":"2024-03-26T17:59:52.461227+0000","last_fullsized":"2024-03-26T17:59:52.461227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:28:39.310849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.518760+0000","last_change":"2024-03-26T17:59:46.432635+0000","last_active":"2024-03-26T18:00:06.518760+0000","last_peered":"2024-03-26T18:00:06.518760+0000","last_clean":"2024-03-26T18:00:06.518760+0000","last_became_active":"2024-03-26T17:59:46.432317+0000","last_became_peered":"2024-03-26T17:59:46.432317+0000","last_unstale":"2024-03-26T18:00:06.518760+0000","last_undegraded":"2024-03-26T18:00:06.518760+0000","last_fullsized":"2024-03-26T18:00:06.518760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:48:15.018906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725253+0000","last_change":"2024-03-26T17:59:51.480678+0000","last_active":"2024-03-26T17:59:52.725253+0000","last_peered":"2024-03-26T17:59:52.725253+0000","last_clean":"2024-03-26T17:59:52.725253+0000","last_became_active":"2024-03-26T17:59:51.480290+0000","last_became_peered":"2024-03-26T17:59:51.480290+0000","last_unstale":"2024-03-26T17:59:52.725253+0000","last_undegraded":"2024-03-26T17:59:52.725253+0000","last_fullsized":"2024-03-26T17:59:52.725253+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:02:20.279963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461932+0000","last_change":"2024-03-26T17:59:46.445569+0000","last_active":"2024-03-26T17:59:52.461932+0000","last_peered":"2024-03-26T17:59:52.461932+0000","last_clean":"2024-03-26T17:59:52.461932+0000","last_became_active":"2024-03-26T17:59:46.445114+0000","last_became_peered":"2024-03-26T17:59:46.445114+0000","last_unstale":"2024-03-26T17:59:52.461932+0000","last_undegraded":"2024-03-26T17:59:52.461932+0000","last_fullsized":"2024-03-26T17:59:52.461932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:18:59.367401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461911+0000","last_change":"2024-03-26T17:59:47.227901+0000","last_active":"2024-03-26T17:59:52.461911+0000","last_peered":"2024-03-26T17:59:52.461911+0000","last_clean":"2024-03-26T17:59:52.461911+0000","last_became_active":"2024-03-26T17:59:47.227336+0000","last_became_peered":"2024-03-26T17:59:47.227336+0000","last_unstale":"2024-03-26T17:59:52.461911+0000","last_undegraded":"2024-03-26T17:59:52.461911+0000","last_fullsized":"2024-03-26T17:59:52.461911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:44:25.326477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724557+0000","last_change":"2024-03-26T17:59:51.482051+0000","last_active":"2024-03-26T17:59:52.724557+0000","last_peered":"2024-03-26T17:59:52.724557+0000","last_clean":"2024-03-26T17:59:52.724557+0000","last_became_active":"2024-03-26T17:59:51.481417+0000","last_became_peered":"2024-03-26T17:59:51.481417+0000","last_unstale":"2024-03-26T17:59:52.724557+0000","last_undegraded":"2024-03-26T17:59:52.724557+0000","last_fullsized":"2024-03-26T17:59:52.724557+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:41:33.294253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725444+0000","last_change":"2024-03-26T17:59:46.436832+0000","last_active":"2024-03-26T17:59:52.725444+0000","last_peered":"2024-03-26T17:59:52.725444+0000","last_clean":"2024-03-26T17:59:52.725444+0000","last_became_active":"2024-03-26T17:59:46.433953+0000","last_became_peered":"2024-03-26T17:59:46.433953+0000","last_unstale":"2024-03-26T17:59:52.725444+0000","last_undegraded":"2024-03-26T17:59:52.725444+0000","last_fullsized":"2024-03-26T17:59:52.725444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:18:30.282277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725997+0000","last_change":"2024-03-26T17:59:47.241212+0000","last_active":"2024-03-26T17:59:52.725997+0000","last_peered":"2024-03-26T17:59:52.725997+0000","last_clean":"2024-03-26T17:59:52.725997+0000","last_became_active":"2024-03-26T17:59:47.240850+0000","last_became_peered":"2024-03-26T17:59:47.240850+0000","last_unstale":"2024-03-26T17:59:52.725997+0000","last_undegraded":"2024-03-26T17:59:52.725997+0000","last_fullsized":"2024-03-26T17:59:52.725997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:25:15.777349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725633+0000","last_change":"2024-03-26T17:59:51.474603+0000","last_active":"2024-03-26T17:59:52.725633+0000","last_peered":"2024-03-26T17:59:52.725633+0000","last_clean":"2024-03-26T17:59:52.725633+0000","last_became_active":"2024-03-26T17:59:51.474069+0000","last_became_peered":"2024-03-26T17:59:51.474069+0000","last_unstale":"2024-03-26T17:59:52.725633+0000","last_undegraded":"2024-03-26T17:59:52.725633+0000","last_fullsized":"2024-03-26T17:59:52.725633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:32:05.459266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.511203+0000","last_change":"2024-03-26T17:59:46.435318+0000","last_active":"2024-03-26T18:00:05.511203+0000","last_peered":"2024-03-26T18:00:05.511203+0000","last_clean":"2024-03-26T18:00:05.511203+0000","last_became_active":"2024-03-26T17:59:46.434982+0000","last_became_peered":"2024-03-26T17:59:46.434982+0000","last_unstale":"2024-03-26T18:00:05.511203+0000","last_undegraded":"2024-03-26T18:00:05.511203+0000","last_fullsized":"2024-03-26T18:00:05.511203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:24:28.364973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726407+0000","last_change":"2024-03-26T17:59:47.239552+0000","last_active":"2024-03-26T17:59:52.726407+0000","last_peered":"2024-03-26T17:59:52.726407+0000","last_clean":"2024-03-26T17:59:52.726407+0000","last_became_active":"2024-03-26T17:59:47.238027+0000","last_became_peered":"2024-03-26T17:59:47.238027+0000","last_unstale":"2024-03-26T17:59:52.726407+0000","last_undegraded":"2024-03-26T17:59:52.726407+0000","last_fullsized":"2024-03-26T17:59:52.726407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:49:39.158517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724351+0000","last_change":"2024-03-26T17:59:51.486887+0000","last_active":"2024-03-26T17:59:52.724351+0000","last_peered":"2024-03-26T17:59:52.724351+0000","last_clean":"2024-03-26T17:59:52.724351+0000","last_became_active":"2024-03-26T17:59:51.486426+0000","last_became_peered":"2024-03-26T17:59:51.486426+0000","last_unstale":"2024-03-26T17:59:52.724351+0000","last_undegraded":"2024-03-26T17:59:52.724351+0000","last_fullsized":"2024-03-26T17:59:52.724351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:56.561561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.512858+0000","last_change":"2024-03-26T17:59:46.437858+0000","last_active":"2024-03-26T18:00:05.512858+0000","last_peered":"2024-03-26T18:00:05.512858+0000","last_clean":"2024-03-26T18:00:05.512858+0000","last_became_active":"2024-03-26T17:59:46.437499+0000","last_became_peered":"2024-03-26T17:59:46.437499+0000","last_unstale":"2024-03-26T18:00:05.512858+0000","last_undegraded":"2024-03-26T18:00:05.512858+0000","last_fullsized":"2024-03-26T18:00:05.512858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:42:01.913000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461449+0000","last_change":"2024-03-26T17:59:47.221959+0000","last_active":"2024-03-26T17:59:52.461449+0000","last_peered":"2024-03-26T17:59:52.461449+0000","last_clean":"2024-03-26T17:59:52.461449+0000","last_became_active":"2024-03-26T17:59:47.221550+0000","last_became_peered":"2024-03-26T17:59:47.221550+0000","last_unstale":"2024-03-26T17:59:52.461449+0000","last_undegraded":"2024-03-26T17:59:52.461449+0000","last_fullsized":"2024-03-26T17:59:52.461449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:53:21.354652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725733+0000","last_change":"2024-03-26T17:59:51.487487+0000","last_active":"2024-03-26T17:59:52.725733+0000","last_peered":"2024-03-26T17:59:52.725733+0000","last_clean":"2024-03-26T17:59:52.725733+0000","last_became_active":"2024-03-26T17:59:51.484494+0000","last_became_peered":"2024-03-26T17:59:51.484494+0000","last_unstale":"2024-03-26T17:59:52.725733+0000","last_undegraded":"2024-03-26T17:59:52.725733+0000","last_fullsized":"2024-03-26T17:59:52.725733+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:29.849739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.512837+0000","last_change":"2024-03-26T17:59:46.437507+0000","last_active":"2024-03-26T18:00:05.512837+0000","last_peered":"2024-03-26T18:00:05.512837+0000","last_clean":"2024-03-26T18:00:05.512837+0000","last_became_active":"2024-03-26T17:59:46.437146+0000","last_became_peered":"2024-03-26T17:59:46.437146+0000","last_unstale":"2024-03-26T18:00:05.512837+0000","last_undegraded":"2024-03-26T18:00:05.512837+0000","last_fullsized":"2024-03-26T18:00:05.512837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:03:44.924632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724636+0000","last_change":"2024-03-26T17:59:47.227678+0000","last_active":"2024-03-26T17:59:52.724636+0000","last_peered":"2024-03-26T17:59:52.724636+0000","last_clean":"2024-03-26T17:59:52.724636+0000","last_became_active":"2024-03-26T17:59:47.224858+0000","last_became_peered":"2024-03-26T17:59:47.224858+0000","last_unstale":"2024-03-26T17:59:52.724636+0000","last_undegraded":"2024-03-26T17:59:52.724636+0000","last_fullsized":"2024-03-26T17:59:52.724636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:39.404469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724491+0000","last_change":"2024-03-26T17:59:51.487800+0000","last_active":"2024-03-26T17:59:52.724491+0000","last_peered":"2024-03-26T17:59:52.724491+0000","last_clean":"2024-03-26T17:59:52.724491+0000","last_became_active":"2024-03-26T17:59:51.486933+0000","last_became_peered":"2024-03-26T17:59:51.486933+0000","last_unstale":"2024-03-26T17:59:52.724491+0000","last_undegraded":"2024-03-26T17:59:52.724491+0000","last_fullsized":"2024-03-26T17:59:52.724491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:05:03.333414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724688+0000","last_change":"2024-03-26T17:59:46.446331+0000","last_active":"2024-03-26T17:59:52.724688+0000","last_peered":"2024-03-26T17:59:52.724688+0000","last_clean":"2024-03-26T17:59:52.724688+0000","last_became_active":"2024-03-26T17:59:46.446013+0000","last_became_peered":"2024-03-26T17:59:46.446013+0000","last_unstale":"2024-03-26T17:59:52.724688+0000","last_undegraded":"2024-03-26T17:59:52.724688+0000","last_fullsized":"2024-03-26T17:59:52.724688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:59:40.458684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725709+0000","last_change":"2024-03-26T17:59:47.236616+0000","last_active":"2024-03-26T17:59:52.725709+0000","last_peered":"2024-03-26T17:59:52.725709+0000","last_clean":"2024-03-26T17:59:52.725709+0000","last_became_active":"2024-03-26T17:59:47.236261+0000","last_became_peered":"2024-03-26T17:59:47.236261+0000","last_unstale":"2024-03-26T17:59:52.725709+0000","last_undegraded":"2024-03-26T17:59:52.725709+0000","last_fullsized":"2024-03-26T17:59:52.725709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:54:45.319703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724401+0000","last_change":"2024-03-26T17:59:51.482500+0000","last_active":"2024-03-26T17:59:52.724401+0000","last_peered":"2024-03-26T17:59:52.724401+0000","last_clean":"2024-03-26T17:59:52.724401+0000","last_became_active":"2024-03-26T17:59:51.482055+0000","last_became_peered":"2024-03-26T17:59:51.482055+0000","last_unstale":"2024-03-26T17:59:52.724401+0000","last_undegraded":"2024-03-26T17:59:52.724401+0000","last_fullsized":"2024-03-26T17:59:52.724401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:27:10.278683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.512933+0000","last_change":"2024-03-26T17:59:46.438600+0000","last_active":"2024-03-26T18:00:05.512933+0000","last_peered":"2024-03-26T18:00:05.512933+0000","last_clean":"2024-03-26T18:00:05.512933+0000","last_became_active":"2024-03-26T17:59:46.438272+0000","last_became_peered":"2024-03-26T17:59:46.438272+0000","last_unstale":"2024-03-26T18:00:05.512933+0000","last_undegraded":"2024-03-26T18:00:05.512933+0000","last_fullsized":"2024-03-26T18:00:05.512933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:18:00.551008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461932+0000","last_change":"2024-03-26T17:59:47.227500+0000","last_active":"2024-03-26T17:59:52.461932+0000","last_peered":"2024-03-26T17:59:52.461932+0000","last_clean":"2024-03-26T17:59:52.461932+0000","last_became_active":"2024-03-26T17:59:47.227184+0000","last_became_peered":"2024-03-26T17:59:47.227184+0000","last_unstale":"2024-03-26T17:59:52.461932+0000","last_undegraded":"2024-03-26T17:59:52.461932+0000","last_fullsized":"2024-03-26T17:59:52.461932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:41:22.876846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724125+0000","last_change":"2024-03-26T17:59:51.479724+0000","last_active":"2024-03-26T17:59:52.724125+0000","last_peered":"2024-03-26T17:59:52.724125+0000","last_clean":"2024-03-26T17:59:52.724125+0000","last_became_active":"2024-03-26T17:59:51.479374+0000","last_became_peered":"2024-03-26T17:59:51.479374+0000","last_unstale":"2024-03-26T17:59:52.724125+0000","last_undegraded":"2024-03-26T17:59:52.724125+0000","last_fullsized":"2024-03-26T17:59:52.724125+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:57:23.213546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725525+0000","last_change":"2024-03-26T17:59:46.442609+0000","last_active":"2024-03-26T17:59:52.725525+0000","last_peered":"2024-03-26T17:59:52.725525+0000","last_clean":"2024-03-26T17:59:52.725525+0000","last_became_active":"2024-03-26T17:59:46.442311+0000","last_became_peered":"2024-03-26T17:59:46.442311+0000","last_unstale":"2024-03-26T17:59:52.725525+0000","last_undegraded":"2024-03-26T17:59:52.725525+0000","last_fullsized":"2024-03-26T17:59:52.725525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:02:40.141449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462370+0000","last_change":"2024-03-26T17:59:47.228152+0000","last_active":"2024-03-26T17:59:52.462370+0000","last_peered":"2024-03-26T17:59:52.462370+0000","last_clean":"2024-03-26T17:59:52.462370+0000","last_became_active":"2024-03-26T17:59:47.227859+0000","last_became_peered":"2024-03-26T17:59:47.227859+0000","last_unstale":"2024-03-26T17:59:52.462370+0000","last_undegraded":"2024-03-26T17:59:52.462370+0000","last_fullsized":"2024-03-26T17:59:52.462370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:16:59.445039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724959+0000","last_change":"2024-03-26T17:59:51.487576+0000","last_active":"2024-03-26T17:59:52.724959+0000","last_peered":"2024-03-26T17:59:52.724959+0000","last_clean":"2024-03-26T17:59:52.724959+0000","last_became_active":"2024-03-26T17:59:51.486995+0000","last_became_peered":"2024-03-26T17:59:51.486995+0000","last_unstale":"2024-03-26T17:59:52.724959+0000","last_undegraded":"2024-03-26T17:59:52.724959+0000","last_fullsized":"2024-03-26T17:59:52.724959+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:58.061260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726185+0000","last_change":"2024-03-26T17:59:46.434433+0000","last_active":"2024-03-26T17:59:52.726185+0000","last_peered":"2024-03-26T17:59:52.726185+0000","last_clean":"2024-03-26T17:59:52.726185+0000","last_became_active":"2024-03-26T17:59:46.434117+0000","last_became_peered":"2024-03-26T17:59:46.434117+0000","last_unstale":"2024-03-26T17:59:52.726185+0000","last_undegraded":"2024-03-26T17:59:52.726185+0000","last_fullsized":"2024-03-26T17:59:52.726185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:28.083880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462067+0000","last_change":"2024-03-26T17:59:47.232406+0000","last_active":"2024-03-26T17:59:52.462067+0000","last_peered":"2024-03-26T17:59:52.462067+0000","last_clean":"2024-03-26T17:59:52.462067+0000","last_became_active":"2024-03-26T17:59:47.232076+0000","last_became_peered":"2024-03-26T17:59:47.232076+0000","last_unstale":"2024-03-26T17:59:52.462067+0000","last_undegraded":"2024-03-26T17:59:52.462067+0000","last_fullsized":"2024-03-26T17:59:52.462067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:46:23.159106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724630+0000","last_change":"2024-03-26T17:59:51.475555+0000","last_active":"2024-03-26T17:59:52.724630+0000","last_peered":"2024-03-26T17:59:52.724630+0000","last_clean":"2024-03-26T17:59:52.724630+0000","last_became_active":"2024-03-26T17:59:51.475136+0000","last_became_peered":"2024-03-26T17:59:51.475136+0000","last_unstale":"2024-03-26T17:59:52.724630+0000","last_undegraded":"2024-03-26T17:59:52.724630+0000","last_fullsized":"2024-03-26T17:59:52.724630+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:28:07.279237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461907+0000","last_change":"2024-03-26T17:59:46.437510+0000","last_active":"2024-03-26T17:59:52.461907+0000","last_peered":"2024-03-26T17:59:52.461907+0000","last_clean":"2024-03-26T17:59:52.461907+0000","last_became_active":"2024-03-26T17:59:46.437191+0000","last_became_peered":"2024-03-26T17:59:46.437191+0000","last_unstale":"2024-03-26T17:59:52.461907+0000","last_undegraded":"2024-03-26T17:59:52.461907+0000","last_fullsized":"2024-03-26T17:59:52.461907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:19:13.759085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462315+0000","last_change":"2024-03-26T17:59:47.225692+0000","last_active":"2024-03-26T17:59:52.462315+0000","last_peered":"2024-03-26T17:59:52.462315+0000","last_clean":"2024-03-26T17:59:52.462315+0000","last_became_active":"2024-03-26T17:59:47.225363+0000","last_became_peered":"2024-03-26T17:59:47.225363+0000","last_unstale":"2024-03-26T17:59:52.462315+0000","last_undegraded":"2024-03-26T17:59:52.462315+0000","last_fullsized":"2024-03-26T17:59:52.462315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:01:44.318833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724774+0000","last_change":"2024-03-26T17:59:47.227155+0000","last_active":"2024-03-26T17:59:52.724774+0000","last_peered":"2024-03-26T17:59:52.724774+0000","last_clean":"2024-03-26T17:59:52.724774+0000","last_became_active":"2024-03-26T17:59:47.226504+0000","last_became_peered":"2024-03-26T17:59:47.226504+0000","last_unstale":"2024-03-26T17:59:52.724774+0000","last_undegraded":"2024-03-26T17:59:52.724774+0000","last_fullsized":"2024-03-26T17:59:52.724774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:16:48.427819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726000+0000","last_change":"2024-03-26T17:59:47.230394+0000","last_active":"2024-03-26T17:59:52.726000+0000","last_peered":"2024-03-26T17:59:52.726000+0000","last_clean":"2024-03-26T17:59:52.726000+0000","last_became_active":"2024-03-26T17:59:47.228367+0000","last_became_peered":"2024-03-26T17:59:47.228367+0000","last_unstale":"2024-03-26T17:59:52.726000+0000","last_undegraded":"2024-03-26T17:59:52.726000+0000","last_fullsized":"2024-03-26T17:59:52.726000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:59:04.158938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724568+0000","last_change":"2024-03-26T17:59:47.230877+0000","last_active":"2024-03-26T17:59:52.724568+0000","last_peered":"2024-03-26T17:59:52.724568+0000","last_clean":"2024-03-26T17:59:52.724568+0000","last_became_active":"2024-03-26T17:59:47.230478+0000","last_became_peered":"2024-03-26T17:59:47.230478+0000","last_unstale":"2024-03-26T17:59:52.724568+0000","last_undegraded":"2024-03-26T17:59:52.724568+0000","last_fullsized":"2024-03-26T17:59:52.724568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:46:25.867718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725182+0000","last_change":"2024-03-26T17:59:47.237012+0000","last_active":"2024-03-26T17:59:52.725182+0000","last_peered":"2024-03-26T17:59:52.725182+0000","last_clean":"2024-03-26T17:59:52.725182+0000","last_became_active":"2024-03-26T17:59:47.236705+0000","last_became_peered":"2024-03-26T17:59:47.236705+0000","last_unstale":"2024-03-26T17:59:52.725182+0000","last_undegraded":"2024-03-26T17:59:52.725182+0000","last_fullsized":"2024-03-26T17:59:52.725182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:34:41.379731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461829+0000","last_change":"2024-03-26T17:59:47.227641+0000","last_active":"2024-03-26T17:59:52.461829+0000","last_peered":"2024-03-26T17:59:52.461829+0000","last_clean":"2024-03-26T17:59:52.461829+0000","last_became_active":"2024-03-26T17:59:47.226921+0000","last_became_peered":"2024-03-26T17:59:47.226921+0000","last_unstale":"2024-03-26T17:59:52.461829+0000","last_undegraded":"2024-03-26T17:59:52.461829+0000","last_fullsized":"2024-03-26T17:59:52.461829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:48.442562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726275+0000","last_change":"2024-03-26T17:59:47.232387+0000","last_active":"2024-03-26T17:59:52.726275+0000","last_peered":"2024-03-26T17:59:52.726275+0000","last_clean":"2024-03-26T17:59:52.726275+0000","last_became_active":"2024-03-26T17:59:47.231934+0000","last_became_peered":"2024-03-26T17:59:47.231934+0000","last_unstale":"2024-03-26T17:59:52.726275+0000","last_undegraded":"2024-03-26T17:59:52.726275+0000","last_fullsized":"2024-03-26T17:59:52.726275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:32.006136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725108+0000","last_change":"2024-03-26T17:59:47.226178+0000","last_active":"2024-03-26T17:59:52.725108+0000","last_peered":"2024-03-26T17:59:52.725108+0000","last_clean":"2024-03-26T17:59:52.725108+0000","last_became_active":"2024-03-26T17:59:47.224692+0000","last_became_peered":"2024-03-26T17:59:47.224692+0000","last_unstale":"2024-03-26T17:59:52.725108+0000","last_undegraded":"2024-03-26T17:59:52.725108+0000","last_fullsized":"2024-03-26T17:59:52.725108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:18.617841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726177+0000","last_change":"2024-03-26T17:59:47.238289+0000","last_active":"2024-03-26T17:59:52.726177+0000","last_peered":"2024-03-26T17:59:52.726177+0000","last_clean":"2024-03-26T17:59:52.726177+0000","last_became_active":"2024-03-26T17:59:47.237904+0000","last_became_peered":"2024-03-26T17:59:47.237904+0000","last_unstale":"2024-03-26T17:59:52.726177+0000","last_undegraded":"2024-03-26T17:59:52.726177+0000","last_fullsized":"2024-03-26T17:59:52.726177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:57:42.695335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726249+0000","last_change":"2024-03-26T17:59:47.234780+0000","last_active":"2024-03-26T17:59:52.726249+0000","last_peered":"2024-03-26T17:59:52.726249+0000","last_clean":"2024-03-26T17:59:52.726249+0000","last_became_active":"2024-03-26T17:59:47.234478+0000","last_became_peered":"2024-03-26T17:59:47.234478+0000","last_unstale":"2024-03-26T17:59:52.726249+0000","last_undegraded":"2024-03-26T17:59:52.726249+0000","last_fullsized":"2024-03-26T17:59:52.726249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:57:43.675947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725390+0000","last_change":"2024-03-26T17:59:47.240057+0000","last_active":"2024-03-26T17:59:52.725390+0000","last_peered":"2024-03-26T17:59:52.725390+0000","last_clean":"2024-03-26T17:59:52.725390+0000","last_became_active":"2024-03-26T17:59:47.239725+0000","last_became_peered":"2024-03-26T17:59:47.239725+0000","last_unstale":"2024-03-26T17:59:52.725390+0000","last_undegraded":"2024-03-26T17:59:52.725390+0000","last_fullsized":"2024-03-26T17:59:52.725390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:45.364403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724433+0000","last_change":"2024-03-26T17:59:47.228059+0000","last_active":"2024-03-26T17:59:52.724433+0000","last_peered":"2024-03-26T17:59:52.724433+0000","last_clean":"2024-03-26T17:59:52.724433+0000","last_became_active":"2024-03-26T17:59:47.227666+0000","last_became_peered":"2024-03-26T17:59:47.227666+0000","last_unstale":"2024-03-26T17:59:52.724433+0000","last_undegraded":"2024-03-26T17:59:52.724433+0000","last_fullsized":"2024-03-26T17:59:52.724433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:19:26.440271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461645+0000","last_change":"2024-03-26T17:59:47.228705+0000","last_active":"2024-03-26T17:59:52.461645+0000","last_peered":"2024-03-26T17:59:52.461645+0000","last_clean":"2024-03-26T17:59:52.461645+0000","last_became_active":"2024-03-26T17:59:47.228425+0000","last_became_peered":"2024-03-26T17:59:47.228425+0000","last_unstale":"2024-03-26T17:59:52.461645+0000","last_undegraded":"2024-03-26T17:59:52.461645+0000","last_fullsized":"2024-03-26T17:59:52.461645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:43:32.055816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724933+0000","last_change":"2024-03-26T17:59:47.232080+0000","last_active":"2024-03-26T17:59:52.724933+0000","last_peered":"2024-03-26T17:59:52.724933+0000","last_clean":"2024-03-26T17:59:52.724933+0000","last_became_active":"2024-03-26T17:59:47.231672+0000","last_became_peered":"2024-03-26T17:59:47.231672+0000","last_unstale":"2024-03-26T17:59:52.724933+0000","last_undegraded":"2024-03-26T17:59:52.724933+0000","last_fullsized":"2024-03-26T17:59:52.724933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:39.962065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250240,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5576,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":92,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":18238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":480266,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612629,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1074670,"data_compressed":50921,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":480848,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":480892,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1072426,"data_compressed":50921,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":484108,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":481744,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":484142,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149753,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":480460,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888116,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":480288,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":1070816,"data_compressed":50921,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34580,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34580,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T18:00:10.302 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T18:00:10.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.464+0000 7f33c1212640 1 -- 172.21.15.49:0/3577525705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33bc102410 msgr2=0x7f33bc108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:10.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.464+0000 7f33c1212640 1 --2- 172.21.15.49:0/3577525705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33bc102410 0x7f33bc108ca0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009ca0 tx=0x7f33b402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.464+0000 7f33c1212640 1 -- 172.21.15.49:0/3577525705 shutdown_connections 2024-03-26T18:00:10.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.464+0000 7f33c1212640 1 --2- 172.21.15.49:0/3577525705 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33bc101a50 0x7f33bc101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.464+0000 7f33c1212640 1 --2- 172.21.15.49:0/3577525705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33bc102410 0x7f33bc108ca0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.464+0000 7f33c1212640 1 --2- 172.21.15.49:0/3577525705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33bc100860 0x7f33bc100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.464+0000 7f33c1212640 1 -- 172.21.15.49:0/3577525705 >> 172.21.15.49:0/3577525705 conn(0x7f33bc0fbdf0 msgr2=0x7f33bc0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:10.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.464+0000 7f33c1212640 1 -- 172.21.15.49:0/3577525705 shutdown_connections 2024-03-26T18:00:10.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.464+0000 7f33c1212640 1 -- 172.21.15.49:0/3577525705 wait complete. 2024-03-26T18:00:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33c1212640 1 Processor -- start 2024-03-26T18:00:10.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33c1212640 1 -- start start 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33c1212640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33bc100860 0x7f33bc06aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33c1212640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33bc101a50 0x7f33bc19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33c1212640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33bc102410 0x7f33bc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33c1212640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc10c470 con 0x7f33bc102410 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33c1212640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc10c2f0 con 0x7f33bc101a50 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33c1212640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc10c5f0 con 0x7f33bc100860 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33bb59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33bc102410 0x7f33bc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33b259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33bc101a50 0x7f33bc19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33bad9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33bc100860 0x7f33bc06aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33bb59e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33bc102410 0x7f33bc19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45682/0 (socket says 172.21.15.49:45682) 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33b259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33bc101a50 0x7f33bc19c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:35118/0 (socket says 172.21.15.49:35118) 2024-03-26T18:00:10.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33bb59e640 1 -- 172.21.15.49:0/2024423864 learned_addr learned my addr 172.21.15.49:0/2024423864 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33b259c640 1 -- 172.21.15.49:0/2024423864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33bc100860 msgr2=0x7f33bc06aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33b259c640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33bc100860 0x7f33bc06aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33b259c640 1 -- 172.21.15.49:0/2024423864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33bc102410 msgr2=0x7f33bc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33b259c640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33bc102410 0x7f33bc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33b259c640 1 -- 172.21.15.49:0/2024423864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc19f7b0 con 0x7f33bc101a50 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.465+0000 7f33bb59e640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33bc102410 0x7f33bc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.466+0000 7f33b259c640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33bc101a50 0x7f33bc19c8b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e9f0 tx=0x7f33a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.466+0000 7f33b8d99640 1 -- 172.21.15.49:0/2024423864 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a400ccc0 con 0x7f33bc101a50 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.466+0000 7f33b8d99640 1 -- 172.21.15.49:0/2024423864 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33a4005020 con 0x7f33bc101a50 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.466+0000 7f33b8d99640 1 -- 172.21.15.49:0/2024423864 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a40049e0 con 0x7f33bc101a50 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.466+0000 7f33c1212640 1 -- 172.21.15.49:0/2024423864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc19faa0 con 0x7f33bc101a50 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.466+0000 7f33c1212640 1 -- 172.21.15.49:0/2024423864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc19ffa0 con 0x7f33bc101a50 2024-03-26T18:00:10.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.467+0000 7f33c1212640 1 -- 172.21.15.49:0/2024423864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33bc107b20 con 0x7f33bc101a50 2024-03-26T18:00:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.468+0000 7f33b8d99640 1 -- 172.21.15.49:0/2024423864 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f33a4005560 con 0x7f33bc101a50 2024-03-26T18:00:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.468+0000 7f33b8d99640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33a80a2fb0 0x7f33a80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.469+0000 7f33bad9d640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33a80a2fb0 0x7f33a80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.469+0000 7f33b8d99640 1 -- 172.21.15.49:0/2024423864 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f33a40cf760 con 0x7f33bc101a50 2024-03-26T18:00:10.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.469+0000 7f33bad9d640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33a80a2fb0 0x7f33a80a5470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f33bc1018b0 tx=0x7f33ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:10.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.472+0000 7f33b8d99640 1 -- 172.21.15.49:0/2024423864 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f33a40051d0 con 0x7f33bc101a50 2024-03-26T18:00:10.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.655+0000 7f33c1212640 1 -- 172.21.15.49:0/2024423864 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33bc061d00 con 0x7f33a80a2fb0 2024-03-26T18:00:10.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.665+0000 7f33b8d99640 1 -- 172.21.15.49:0/2024423864 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340512 (secure 0 0 0) 0x7f33bc061d00 con 0x7f33a80a2fb0 2024-03-26T18:00:10.652 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:00:10.654 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-26T18:00:10.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 -- 172.21.15.49:0/2024423864 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33a80a2fb0 msgr2=0x7f33a80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:10.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33a80a2fb0 0x7f33a80a5470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f33bc1018b0 tx=0x7f33ac005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 -- 172.21.15.49:0/2024423864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33bc101a50 msgr2=0x7f33bc19c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33bc101a50 0x7f33bc19c8b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e9f0 tx=0x7f33a400eeb0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 -- 172.21.15.49:0/2024423864 shutdown_connections 2024-03-26T18:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f33a80a2fb0 0x7f33a80a5470 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33bc102410 0x7f33bc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33bc101a50 0x7f33bc19c8b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 --2- 172.21.15.49:0/2024423864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33bc100860 0x7f33bc06aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 -- 172.21.15.49:0/2024423864 >> 172.21.15.49:0/2024423864 conn(0x7f33bc0fbdf0 msgr2=0x7f33bc104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 -- 172.21.15.49:0/2024423864 shutdown_connections 2024-03-26T18:00:10.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:10.668+0000 7f33c1212640 1 -- 172.21.15.49:0/2024423864 wait complete. 2024-03-26T18:00:10.661 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:10 smithi049 ceph-mon[32331]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-03-26T18:00:10.662 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:10 smithi049 ceph-mon[32331]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:10.662 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:10 smithi049 ceph-mon[32331]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:10.662 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:10 smithi049 ceph-mon[32331]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T18:00:10.662 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:10 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/479878660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:00:10.662 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:10 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2080088516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:10.663 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-03-26T18:00:08.707371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595944,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":137,"num_read_kb":120,"num_write":273,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360844,"kb_used_data":11664,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580980,"statfs":{"total":1151940427776,"available":1151570923520,"internally_reserved":0,"allocated":11943936,"data_stored":7551016,"data_compressed":285108,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3166,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":47,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002952"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725719+0000","last_change":"2024-03-26T17:59:47.232887+0000","last_active":"2024-03-26T17:59:52.725719+0000","last_peered":"2024-03-26T17:59:52.725719+0000","last_clean":"2024-03-26T17:59:52.725719+0000","last_became_active":"2024-03-26T17:59:47.232559+0000","last_became_peered":"2024-03-26T17:59:47.232559+0000","last_unstale":"2024-03-26T17:59:52.725719+0000","last_undegraded":"2024-03-26T17:59:52.725719+0000","last_fullsized":"2024-03-26T17:59:52.725719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:29:01.759592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726505+0000","last_change":"2024-03-26T17:59:47.230719+0000","last_active":"2024-03-26T17:59:52.726505+0000","last_peered":"2024-03-26T17:59:52.726505+0000","last_clean":"2024-03-26T17:59:52.726505+0000","last_became_active":"2024-03-26T17:59:47.230411+0000","last_became_peered":"2024-03-26T17:59:47.230411+0000","last_unstale":"2024-03-26T17:59:52.726505+0000","last_undegraded":"2024-03-26T17:59:52.726505+0000","last_fullsized":"2024-03-26T17:59:52.726505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:29:21.410798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726213+0000","last_change":"2024-03-26T17:59:47.233625+0000","last_active":"2024-03-26T17:59:52.726213+0000","last_peered":"2024-03-26T17:59:52.726213+0000","last_clean":"2024-03-26T17:59:52.726213+0000","last_became_active":"2024-03-26T17:59:47.233318+0000","last_became_peered":"2024-03-26T17:59:47.233318+0000","last_unstale":"2024-03-26T17:59:52.726213+0000","last_undegraded":"2024-03-26T17:59:52.726213+0000","last_fullsized":"2024-03-26T17:59:52.726213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:53:08.524508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724823+0000","last_change":"2024-03-26T17:59:47.234380+0000","last_active":"2024-03-26T17:59:52.724823+0000","last_peered":"2024-03-26T17:59:52.724823+0000","last_clean":"2024-03-26T17:59:52.724823+0000","last_became_active":"2024-03-26T17:59:47.233804+0000","last_became_peered":"2024-03-26T17:59:47.233804+0000","last_unstale":"2024-03-26T17:59:52.724823+0000","last_undegraded":"2024-03-26T17:59:52.724823+0000","last_fullsized":"2024-03-26T17:59:52.724823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:26:59.956704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461492+0000","last_change":"2024-03-26T17:59:47.232523+0000","last_active":"2024-03-26T17:59:52.461492+0000","last_peered":"2024-03-26T17:59:52.461492+0000","last_clean":"2024-03-26T17:59:52.461492+0000","last_became_active":"2024-03-26T17:59:47.232183+0000","last_became_peered":"2024-03-26T17:59:47.232183+0000","last_unstale":"2024-03-26T17:59:52.461492+0000","last_undegraded":"2024-03-26T17:59:52.461492+0000","last_fullsized":"2024-03-26T17:59:52.461492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:47:06.538092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.668953+0000","last_change":"2024-03-26T17:59:47.229078+0000","last_active":"2024-03-26T17:59:52.668953+0000","last_peered":"2024-03-26T17:59:52.668953+0000","last_clean":"2024-03-26T17:59:52.668953+0000","last_became_active":"2024-03-26T17:59:47.228757+0000","last_became_peered":"2024-03-26T17:59:47.228757+0000","last_unstale":"2024-03-26T17:59:52.668953+0000","last_undegraded":"2024-03-26T17:59:52.668953+0000","last_fullsized":"2024-03-26T17:59:52.668953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:41:32.459219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461723+0000","last_change":"2024-03-26T17:59:47.232111+0000","last_active":"2024-03-26T17:59:52.461723+0000","last_peered":"2024-03-26T17:59:52.461723+0000","last_clean":"2024-03-26T17:59:52.461723+0000","last_became_active":"2024-03-26T17:59:47.231440+0000","last_became_peered":"2024-03-26T17:59:47.231440+0000","last_unstale":"2024-03-26T17:59:52.461723+0000","last_undegraded":"2024-03-26T17:59:52.461723+0000","last_fullsized":"2024-03-26T17:59:52.461723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:23:11.607084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724386+0000","last_change":"2024-03-26T17:59:47.231875+0000","last_active":"2024-03-26T17:59:52.724386+0000","last_peered":"2024-03-26T17:59:52.724386+0000","last_clean":"2024-03-26T17:59:52.724386+0000","last_became_active":"2024-03-26T17:59:47.231534+0000","last_became_peered":"2024-03-26T17:59:47.231534+0000","last_unstale":"2024-03-26T17:59:52.724386+0000","last_undegraded":"2024-03-26T17:59:52.724386+0000","last_fullsized":"2024-03-26T17:59:52.724386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:05:33.010733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724633+0000","last_change":"2024-03-26T17:59:47.235321+0000","last_active":"2024-03-26T17:59:52.724633+0000","last_peered":"2024-03-26T17:59:52.724633+0000","last_clean":"2024-03-26T17:59:52.724633+0000","last_became_active":"2024-03-26T17:59:47.231778+0000","last_became_peered":"2024-03-26T17:59:47.231778+0000","last_unstale":"2024-03-26T17:59:52.724633+0000","last_undegraded":"2024-03-26T17:59:52.724633+0000","last_fullsized":"2024-03-26T17:59:52.724633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:58:25.613249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.669258+0000","last_change":"2024-03-26T17:59:47.224272+0000","last_active":"2024-03-26T17:59:52.669258+0000","last_peered":"2024-03-26T17:59:52.669258+0000","last_clean":"2024-03-26T17:59:52.669258+0000","last_became_active":"2024-03-26T17:59:47.223439+0000","last_became_peered":"2024-03-26T17:59:47.223439+0000","last_unstale":"2024-03-26T17:59:52.669258+0000","last_undegraded":"2024-03-26T17:59:52.669258+0000","last_fullsized":"2024-03-26T17:59:52.669258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:28:40.428780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462312+0000","last_change":"2024-03-26T17:59:47.227589+0000","last_active":"2024-03-26T17:59:52.462312+0000","last_peered":"2024-03-26T17:59:52.462312+0000","last_clean":"2024-03-26T17:59:52.462312+0000","last_became_active":"2024-03-26T17:59:47.227134+0000","last_became_peered":"2024-03-26T17:59:47.227134+0000","last_unstale":"2024-03-26T17:59:52.462312+0000","last_undegraded":"2024-03-26T17:59:52.462312+0000","last_fullsized":"2024-03-26T17:59:52.462312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:39:33.093899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724741+0000","last_change":"2024-03-26T17:59:47.233329+0000","last_active":"2024-03-26T17:59:52.724741+0000","last_peered":"2024-03-26T17:59:52.724741+0000","last_clean":"2024-03-26T17:59:52.724741+0000","last_became_active":"2024-03-26T17:59:47.233039+0000","last_became_peered":"2024-03-26T17:59:47.233039+0000","last_unstale":"2024-03-26T17:59:52.724741+0000","last_undegraded":"2024-03-26T17:59:52.724741+0000","last_fullsized":"2024-03-26T17:59:52.724741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:35:43.280834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461993+0000","last_change":"2024-03-26T17:59:47.234468+0000","last_active":"2024-03-26T17:59:52.461993+0000","last_peered":"2024-03-26T17:59:52.461993+0000","last_clean":"2024-03-26T17:59:52.461993+0000","last_became_active":"2024-03-26T17:59:47.234063+0000","last_became_peered":"2024-03-26T17:59:47.234063+0000","last_unstale":"2024-03-26T17:59:52.461993+0000","last_undegraded":"2024-03-26T17:59:52.461993+0000","last_fullsized":"2024-03-26T17:59:52.461993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:25.542893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724869+0000","last_change":"2024-03-26T17:59:47.229076+0000","last_active":"2024-03-26T17:59:52.724869+0000","last_peered":"2024-03-26T17:59:52.724869+0000","last_clean":"2024-03-26T17:59:52.724869+0000","last_became_active":"2024-03-26T17:59:47.228719+0000","last_became_peered":"2024-03-26T17:59:47.228719+0000","last_unstale":"2024-03-26T17:59:52.724869+0000","last_undegraded":"2024-03-26T17:59:52.724869+0000","last_fullsized":"2024-03-26T17:59:52.724869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:03:15.085812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724835+0000","last_change":"2024-03-26T17:59:47.239166+0000","last_active":"2024-03-26T17:59:52.724835+0000","last_peered":"2024-03-26T17:59:52.724835+0000","last_clean":"2024-03-26T17:59:52.724835+0000","last_became_active":"2024-03-26T17:59:47.236051+0000","last_became_peered":"2024-03-26T17:59:47.236051+0000","last_unstale":"2024-03-26T17:59:52.724835+0000","last_undegraded":"2024-03-26T17:59:52.724835+0000","last_fullsized":"2024-03-26T17:59:52.724835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:46:36.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.723905+0000","last_change":"2024-03-26T17:59:47.231246+0000","last_active":"2024-03-26T17:59:52.723905+0000","last_peered":"2024-03-26T17:59:52.723905+0000","last_clean":"2024-03-26T17:59:52.723905+0000","last_became_active":"2024-03-26T17:59:47.228587+0000","last_became_peered":"2024-03-26T17:59:47.228587+0000","last_unstale":"2024-03-26T17:59:52.723905+0000","last_undegraded":"2024-03-26T17:59:52.723905+0000","last_fullsized":"2024-03-26T17:59:52.723905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:02:25.850361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725614+0000","last_change":"2024-03-26T17:59:47.234790+0000","last_active":"2024-03-26T17:59:52.725614+0000","last_peered":"2024-03-26T17:59:52.725614+0000","last_clean":"2024-03-26T17:59:52.725614+0000","last_became_active":"2024-03-26T17:59:47.234436+0000","last_became_peered":"2024-03-26T17:59:47.234436+0000","last_unstale":"2024-03-26T17:59:52.725614+0000","last_undegraded":"2024-03-26T17:59:52.725614+0000","last_fullsized":"2024-03-26T17:59:52.725614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:54:26.949089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725352+0000","last_change":"2024-03-26T17:59:47.238064+0000","last_active":"2024-03-26T17:59:52.725352+0000","last_peered":"2024-03-26T17:59:52.725352+0000","last_clean":"2024-03-26T17:59:52.725352+0000","last_became_active":"2024-03-26T17:59:47.237754+0000","last_became_peered":"2024-03-26T17:59:47.237754+0000","last_unstale":"2024-03-26T17:59:52.725352+0000","last_undegraded":"2024-03-26T17:59:52.725352+0000","last_fullsized":"2024-03-26T17:59:52.725352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:27:29.603516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725383+0000","last_change":"2024-03-26T17:59:47.231000+0000","last_active":"2024-03-26T17:59:52.725383+0000","last_peered":"2024-03-26T17:59:52.725383+0000","last_clean":"2024-03-26T17:59:52.725383+0000","last_became_active":"2024-03-26T17:59:47.230675+0000","last_became_peered":"2024-03-26T17:59:47.230675+0000","last_unstale":"2024-03-26T17:59:52.725383+0000","last_undegraded":"2024-03-26T17:59:52.725383+0000","last_fullsized":"2024-03-26T17:59:52.725383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:56:59.839962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725877+0000","last_change":"2024-03-26T17:59:51.480711+0000","last_active":"2024-03-26T17:59:52.725877+0000","last_peered":"2024-03-26T17:59:52.725877+0000","last_clean":"2024-03-26T17:59:52.725877+0000","last_became_active":"2024-03-26T17:59:51.480111+0000","last_became_peered":"2024-03-26T17:59:51.480111+0000","last_unstale":"2024-03-26T17:59:52.725877+0000","last_undegraded":"2024-03-26T17:59:52.725877+0000","last_fullsized":"2024-03-26T17:59:52.725877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:40:55.456138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725787+0000","last_change":"2024-03-26T17:59:46.434642+0000","last_active":"2024-03-26T17:59:52.725787+0000","last_peered":"2024-03-26T17:59:52.725787+0000","last_clean":"2024-03-26T17:59:52.725787+0000","last_became_active":"2024-03-26T17:59:46.434209+0000","last_became_peered":"2024-03-26T17:59:46.434209+0000","last_unstale":"2024-03-26T17:59:52.725787+0000","last_undegraded":"2024-03-26T17:59:52.725787+0000","last_fullsized":"2024-03-26T17:59:52.725787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:55:05.365712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725909+0000","last_change":"2024-03-26T17:59:47.239356+0000","last_active":"2024-03-26T17:59:52.725909+0000","last_peered":"2024-03-26T17:59:52.725909+0000","last_clean":"2024-03-26T17:59:52.725909+0000","last_became_active":"2024-03-26T17:59:47.239020+0000","last_became_peered":"2024-03-26T17:59:47.239020+0000","last_unstale":"2024-03-26T17:59:52.725909+0000","last_undegraded":"2024-03-26T17:59:52.725909+0000","last_fullsized":"2024-03-26T17:59:52.725909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:56.992379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724009+0000","last_change":"2024-03-26T17:59:51.486894+0000","last_active":"2024-03-26T17:59:52.724009+0000","last_peered":"2024-03-26T17:59:52.724009+0000","last_clean":"2024-03-26T17:59:52.724009+0000","last_became_active":"2024-03-26T17:59:51.486465+0000","last_became_peered":"2024-03-26T17:59:51.486465+0000","last_unstale":"2024-03-26T17:59:52.724009+0000","last_undegraded":"2024-03-26T17:59:52.724009+0000","last_fullsized":"2024-03-26T17:59:52.724009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:15:10.145415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726190+0000","last_change":"2024-03-26T17:59:46.437908+0000","last_active":"2024-03-26T17:59:52.726190+0000","last_peered":"2024-03-26T17:59:52.726190+0000","last_clean":"2024-03-26T17:59:52.726190+0000","last_became_active":"2024-03-26T17:59:46.437592+0000","last_became_peered":"2024-03-26T17:59:46.437592+0000","last_unstale":"2024-03-26T17:59:52.726190+0000","last_undegraded":"2024-03-26T17:59:52.726190+0000","last_fullsized":"2024-03-26T17:59:52.726190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:05.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726428+0000","last_change":"2024-03-26T17:59:47.240291+0000","last_active":"2024-03-26T17:59:52.726428+0000","last_peered":"2024-03-26T17:59:52.726428+0000","last_clean":"2024-03-26T17:59:52.726428+0000","last_became_active":"2024-03-26T17:59:47.239893+0000","last_became_peered":"2024-03-26T17:59:47.239893+0000","last_unstale":"2024-03-26T17:59:52.726428+0000","last_undegraded":"2024-03-26T17:59:52.726428+0000","last_fullsized":"2024-03-26T17:59:52.726428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:44:22.946199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724179+0000","last_change":"2024-03-26T17:59:51.483970+0000","last_active":"2024-03-26T17:59:52.724179+0000","last_peered":"2024-03-26T17:59:52.724179+0000","last_clean":"2024-03-26T17:59:52.724179+0000","last_became_active":"2024-03-26T17:59:51.483585+0000","last_became_peered":"2024-03-26T17:59:51.483585+0000","last_unstale":"2024-03-26T17:59:52.724179+0000","last_undegraded":"2024-03-26T17:59:52.724179+0000","last_fullsized":"2024-03-26T17:59:52.724179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:22:30.873421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.519270+0000","last_change":"2024-03-26T17:59:46.439374+0000","last_active":"2024-03-26T18:00:06.519270+0000","last_peered":"2024-03-26T18:00:06.519270+0000","last_clean":"2024-03-26T18:00:06.519270+0000","last_became_active":"2024-03-26T17:59:46.439065+0000","last_became_peered":"2024-03-26T17:59:46.439065+0000","last_unstale":"2024-03-26T18:00:06.519270+0000","last_undegraded":"2024-03-26T18:00:06.519270+0000","last_fullsized":"2024-03-26T18:00:06.519270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:45:37.954375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724501+0000","last_change":"2024-03-26T17:59:47.237168+0000","last_active":"2024-03-26T17:59:52.724501+0000","last_peered":"2024-03-26T17:59:52.724501+0000","last_clean":"2024-03-26T17:59:52.724501+0000","last_became_active":"2024-03-26T17:59:47.232573+0000","last_became_peered":"2024-03-26T17:59:47.232573+0000","last_unstale":"2024-03-26T17:59:52.724501+0000","last_undegraded":"2024-03-26T17:59:52.724501+0000","last_fullsized":"2024-03-26T17:59:52.724501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:00:30.236249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724004+0000","last_change":"2024-03-26T17:59:51.477500+0000","last_active":"2024-03-26T17:59:52.724004+0000","last_peered":"2024-03-26T17:59:52.724004+0000","last_clean":"2024-03-26T17:59:52.724004+0000","last_became_active":"2024-03-26T17:59:51.477064+0000","last_became_peered":"2024-03-26T17:59:51.477064+0000","last_unstale":"2024-03-26T17:59:52.724004+0000","last_undegraded":"2024-03-26T17:59:52.724004+0000","last_fullsized":"2024-03-26T17:59:52.724004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:07:05.265910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461791+0000","last_change":"2024-03-26T17:59:46.437738+0000","last_active":"2024-03-26T17:59:52.461791+0000","last_peered":"2024-03-26T17:59:52.461791+0000","last_clean":"2024-03-26T17:59:52.461791+0000","last_became_active":"2024-03-26T17:59:46.437373+0000","last_became_peered":"2024-03-26T17:59:46.437373+0000","last_unstale":"2024-03-26T17:59:52.461791+0000","last_undegraded":"2024-03-26T17:59:52.461791+0000","last_fullsized":"2024-03-26T17:59:52.461791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:49:32.700437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726060+0000","last_change":"2024-03-26T17:59:47.238169+0000","last_active":"2024-03-26T17:59:52.726060+0000","last_peered":"2024-03-26T17:59:52.726060+0000","last_clean":"2024-03-26T17:59:52.726060+0000","last_became_active":"2024-03-26T17:59:47.237854+0000","last_became_peered":"2024-03-26T17:59:47.237854+0000","last_unstale":"2024-03-26T17:59:52.726060+0000","last_undegraded":"2024-03-26T17:59:52.726060+0000","last_fullsized":"2024-03-26T17:59:52.726060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:23:09.040051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725185+0000","last_change":"2024-03-26T17:59:51.479332+0000","last_active":"2024-03-26T17:59:52.725185+0000","last_peered":"2024-03-26T17:59:52.725185+0000","last_clean":"2024-03-26T17:59:52.725185+0000","last_became_active":"2024-03-26T17:59:51.478857+0000","last_became_peered":"2024-03-26T17:59:51.478857+0000","last_unstale":"2024-03-26T17:59:52.725185+0000","last_undegraded":"2024-03-26T17:59:52.725185+0000","last_fullsized":"2024-03-26T17:59:52.725185+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:04:30.568741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725566+0000","last_change":"2024-03-26T17:59:46.438017+0000","last_active":"2024-03-26T17:59:52.725566+0000","last_peered":"2024-03-26T17:59:52.725566+0000","last_clean":"2024-03-26T17:59:52.725566+0000","last_became_active":"2024-03-26T17:59:46.437625+0000","last_became_peered":"2024-03-26T17:59:46.437625+0000","last_unstale":"2024-03-26T17:59:52.725566+0000","last_undegraded":"2024-03-26T17:59:52.725566+0000","last_fullsized":"2024-03-26T17:59:52.725566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:33:27.460626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461982+0000","last_change":"2024-03-26T17:59:47.232764+0000","last_active":"2024-03-26T17:59:52.461982+0000","last_peered":"2024-03-26T17:59:52.461982+0000","last_clean":"2024-03-26T17:59:52.461982+0000","last_became_active":"2024-03-26T17:59:47.231672+0000","last_became_peered":"2024-03-26T17:59:47.231672+0000","last_unstale":"2024-03-26T17:59:52.461982+0000","last_undegraded":"2024-03-26T17:59:52.461982+0000","last_fullsized":"2024-03-26T17:59:52.461982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.812245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725397+0000","last_change":"2024-03-26T17:59:51.476645+0000","last_active":"2024-03-26T17:59:52.725397+0000","last_peered":"2024-03-26T17:59:52.725397+0000","last_clean":"2024-03-26T17:59:52.725397+0000","last_became_active":"2024-03-26T17:59:51.476232+0000","last_became_peered":"2024-03-26T17:59:51.476232+0000","last_unstale":"2024-03-26T17:59:52.725397+0000","last_undegraded":"2024-03-26T17:59:52.725397+0000","last_fullsized":"2024-03-26T17:59:52.725397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:27:16.109125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.668891+0000","last_change":"2024-03-26T17:59:46.438232+0000","last_active":"2024-03-26T17:59:52.668891+0000","last_peered":"2024-03-26T17:59:52.668891+0000","last_clean":"2024-03-26T17:59:52.668891+0000","last_became_active":"2024-03-26T17:59:46.437871+0000","last_became_peered":"2024-03-26T17:59:46.437871+0000","last_unstale":"2024-03-26T17:59:52.668891+0000","last_undegraded":"2024-03-26T17:59:52.668891+0000","last_fullsized":"2024-03-26T17:59:52.668891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:08.626264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725969+0000","last_change":"2024-03-26T17:59:47.237144+0000","last_active":"2024-03-26T17:59:52.725969+0000","last_peered":"2024-03-26T17:59:52.725969+0000","last_clean":"2024-03-26T17:59:52.725969+0000","last_became_active":"2024-03-26T17:59:47.236696+0000","last_became_peered":"2024-03-26T17:59:47.236696+0000","last_unstale":"2024-03-26T17:59:52.725969+0000","last_undegraded":"2024-03-26T17:59:52.725969+0000","last_fullsized":"2024-03-26T17:59:52.725969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:49:31.197898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725109+0000","last_change":"2024-03-26T17:59:51.490204+0000","last_active":"2024-03-26T17:59:52.725109+0000","last_peered":"2024-03-26T17:59:52.725109+0000","last_clean":"2024-03-26T17:59:52.725109+0000","last_became_active":"2024-03-26T17:59:51.489846+0000","last_became_peered":"2024-03-26T17:59:51.489846+0000","last_unstale":"2024-03-26T17:59:52.725109+0000","last_undegraded":"2024-03-26T17:59:52.725109+0000","last_fullsized":"2024-03-26T17:59:52.725109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:26:12.284436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.520990+0000","last_change":"2024-03-26T17:59:46.429699+0000","last_active":"2024-03-26T18:00:06.520990+0000","last_peered":"2024-03-26T18:00:06.520990+0000","last_clean":"2024-03-26T18:00:06.520990+0000","last_became_active":"2024-03-26T17:59:46.429319+0000","last_became_peered":"2024-03-26T17:59:46.429319+0000","last_unstale":"2024-03-26T18:00:06.520990+0000","last_undegraded":"2024-03-26T18:00:06.520990+0000","last_fullsized":"2024-03-26T18:00:06.520990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:05:15.695696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724666+0000","last_change":"2024-03-26T17:59:47.239705+0000","last_active":"2024-03-26T17:59:52.724666+0000","last_peered":"2024-03-26T17:59:52.724666+0000","last_clean":"2024-03-26T17:59:52.724666+0000","last_became_active":"2024-03-26T17:59:47.239046+0000","last_became_peered":"2024-03-26T17:59:47.239046+0000","last_unstale":"2024-03-26T17:59:52.724666+0000","last_undegraded":"2024-03-26T17:59:52.724666+0000","last_fullsized":"2024-03-26T17:59:52.724666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:08:34.930460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726435+0000","last_change":"2024-03-26T17:59:51.477511+0000","last_active":"2024-03-26T17:59:52.726435+0000","last_peered":"2024-03-26T17:59:52.726435+0000","last_clean":"2024-03-26T17:59:52.726435+0000","last_became_active":"2024-03-26T17:59:51.473949+0000","last_became_peered":"2024-03-26T17:59:51.473949+0000","last_unstale":"2024-03-26T17:59:52.726435+0000","last_undegraded":"2024-03-26T17:59:52.726435+0000","last_fullsized":"2024-03-26T17:59:52.726435+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:26:01.431977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726138+0000","last_change":"2024-03-26T17:59:46.433235+0000","last_active":"2024-03-26T17:59:52.726138+0000","last_peered":"2024-03-26T17:59:52.726138+0000","last_clean":"2024-03-26T17:59:52.726138+0000","last_became_active":"2024-03-26T17:59:46.432929+0000","last_became_peered":"2024-03-26T17:59:46.432929+0000","last_unstale":"2024-03-26T17:59:52.726138+0000","last_undegraded":"2024-03-26T17:59:52.726138+0000","last_fullsized":"2024-03-26T17:59:52.726138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:46:23.064447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724317+0000","last_change":"2024-03-26T17:59:47.232526+0000","last_active":"2024-03-26T17:59:52.724317+0000","last_peered":"2024-03-26T17:59:52.724317+0000","last_clean":"2024-03-26T17:59:52.724317+0000","last_became_active":"2024-03-26T17:59:47.232216+0000","last_became_peered":"2024-03-26T17:59:47.232216+0000","last_unstale":"2024-03-26T17:59:52.724317+0000","last_undegraded":"2024-03-26T17:59:52.724317+0000","last_fullsized":"2024-03-26T17:59:52.724317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:20:07.510565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725141+0000","last_change":"2024-03-26T17:59:51.482706+0000","last_active":"2024-03-26T17:59:52.725141+0000","last_peered":"2024-03-26T17:59:52.725141+0000","last_clean":"2024-03-26T17:59:52.725141+0000","last_became_active":"2024-03-26T17:59:51.482312+0000","last_became_peered":"2024-03-26T17:59:51.482312+0000","last_unstale":"2024-03-26T17:59:52.725141+0000","last_undegraded":"2024-03-26T17:59:52.725141+0000","last_fullsized":"2024-03-26T17:59:52.725141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:42:26.614170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.669151+0000","last_change":"2024-03-26T17:59:46.440919+0000","last_active":"2024-03-26T17:59:52.669151+0000","last_peered":"2024-03-26T17:59:52.669151+0000","last_clean":"2024-03-26T17:59:52.669151+0000","last_became_active":"2024-03-26T17:59:46.440579+0000","last_became_peered":"2024-03-26T17:59:46.440579+0000","last_unstale":"2024-03-26T17:59:52.669151+0000","last_undegraded":"2024-03-26T17:59:52.669151+0000","last_fullsized":"2024-03-26T17:59:52.669151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:38:27.325478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724495+0000","last_change":"2024-03-26T17:59:47.236251+0000","last_active":"2024-03-26T17:59:52.724495+0000","last_peered":"2024-03-26T17:59:52.724495+0000","last_clean":"2024-03-26T17:59:52.724495+0000","last_became_active":"2024-03-26T17:59:47.235941+0000","last_became_peered":"2024-03-26T17:59:47.235941+0000","last_unstale":"2024-03-26T17:59:52.724495+0000","last_undegraded":"2024-03-26T17:59:52.724495+0000","last_fullsized":"2024-03-26T17:59:52.724495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:50:34.821242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724238+0000","last_change":"2024-03-26T17:59:51.480155+0000","last_active":"2024-03-26T17:59:52.724238+0000","last_peered":"2024-03-26T17:59:52.724238+0000","last_clean":"2024-03-26T17:59:52.724238+0000","last_became_active":"2024-03-26T17:59:51.479761+0000","last_became_peered":"2024-03-26T17:59:51.479761+0000","last_unstale":"2024-03-26T17:59:52.724238+0000","last_undegraded":"2024-03-26T17:59:52.724238+0000","last_fullsized":"2024-03-26T17:59:52.724238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:03:44.256586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.510913+0000","last_change":"2024-03-26T17:59:46.436559+0000","last_active":"2024-03-26T18:00:05.510913+0000","last_peered":"2024-03-26T18:00:05.510913+0000","last_clean":"2024-03-26T18:00:05.510913+0000","last_became_active":"2024-03-26T17:59:46.436224+0000","last_became_peered":"2024-03-26T17:59:46.436224+0000","last_unstale":"2024-03-26T18:00:05.510913+0000","last_undegraded":"2024-03-26T18:00:05.510913+0000","last_fullsized":"2024-03-26T18:00:05.510913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:12:56.716278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726538+0000","last_change":"2024-03-26T17:59:47.240266+0000","last_active":"2024-03-26T17:59:52.726538+0000","last_peered":"2024-03-26T17:59:52.726538+0000","last_clean":"2024-03-26T17:59:52.726538+0000","last_became_active":"2024-03-26T17:59:47.238847+0000","last_became_peered":"2024-03-26T17:59:47.238847+0000","last_unstale":"2024-03-26T17:59:52.726538+0000","last_undegraded":"2024-03-26T17:59:52.726538+0000","last_fullsized":"2024-03-26T17:59:52.726538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:47:36.325585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724840+0000","last_change":"2024-03-26T17:59:51.487207+0000","last_active":"2024-03-26T17:59:52.724840+0000","last_peered":"2024-03-26T17:59:52.724840+0000","last_clean":"2024-03-26T17:59:52.724840+0000","last_became_active":"2024-03-26T17:59:51.486351+0000","last_became_peered":"2024-03-26T17:59:51.486351+0000","last_unstale":"2024-03-26T17:59:52.724840+0000","last_undegraded":"2024-03-26T17:59:52.724840+0000","last_fullsized":"2024-03-26T17:59:52.724840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:35:03.445677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725874+0000","last_change":"2024-03-26T17:59:46.437839+0000","last_active":"2024-03-26T17:59:52.725874+0000","last_peered":"2024-03-26T17:59:52.725874+0000","last_clean":"2024-03-26T17:59:52.725874+0000","last_became_active":"2024-03-26T17:59:46.437515+0000","last_became_peered":"2024-03-26T17:59:46.437515+0000","last_unstale":"2024-03-26T17:59:52.725874+0000","last_undegraded":"2024-03-26T17:59:52.725874+0000","last_fullsized":"2024-03-26T17:59:52.725874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:38:34.875841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724596+0000","last_change":"2024-03-26T17:59:47.234833+0000","last_active":"2024-03-26T17:59:52.724596+0000","last_peered":"2024-03-26T17:59:52.724596+0000","last_clean":"2024-03-26T17:59:52.724596+0000","last_became_active":"2024-03-26T17:59:47.232925+0000","last_became_peered":"2024-03-26T17:59:47.232925+0000","last_unstale":"2024-03-26T17:59:52.724596+0000","last_undegraded":"2024-03-26T17:59:52.724596+0000","last_fullsized":"2024-03-26T17:59:52.724596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:42:23.618396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724699+0000","last_change":"2024-03-26T17:59:51.481550+0000","last_active":"2024-03-26T17:59:52.724699+0000","last_peered":"2024-03-26T17:59:52.724699+0000","last_clean":"2024-03-26T17:59:52.724699+0000","last_became_active":"2024-03-26T17:59:51.481142+0000","last_became_peered":"2024-03-26T17:59:51.481142+0000","last_unstale":"2024-03-26T17:59:52.724699+0000","last_undegraded":"2024-03-26T17:59:52.724699+0000","last_fullsized":"2024-03-26T17:59:52.724699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:37.529859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'10","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:03.132117+0000","last_change":"2024-03-26T17:59:46.438936+0000","last_active":"2024-03-26T18:00:03.132117+0000","last_peered":"2024-03-26T18:00:03.132117+0000","last_clean":"2024-03-26T18:00:03.132117+0000","last_became_active":"2024-03-26T17:59:46.438609+0000","last_became_peered":"2024-03-26T17:59:46.438609+0000","last_unstale":"2024-03-26T18:00:03.132117+0000","last_undegraded":"2024-03-26T18:00:03.132117+0000","last_fullsized":"2024-03-26T18:00:03.132117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:38:36.333660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461848+0000","last_change":"2024-03-26T17:59:47.224495+0000","last_active":"2024-03-26T17:59:52.461848+0000","last_peered":"2024-03-26T17:59:52.461848+0000","last_clean":"2024-03-26T17:59:52.461848+0000","last_became_active":"2024-03-26T17:59:47.224184+0000","last_became_peered":"2024-03-26T17:59:47.224184+0000","last_unstale":"2024-03-26T17:59:52.461848+0000","last_undegraded":"2024-03-26T17:59:52.461848+0000","last_fullsized":"2024-03-26T17:59:52.461848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:17:23.028748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726584+0000","last_change":"2024-03-26T17:59:51.483006+0000","last_active":"2024-03-26T17:59:52.726584+0000","last_peered":"2024-03-26T17:59:52.726584+0000","last_clean":"2024-03-26T17:59:52.726584+0000","last_became_active":"2024-03-26T17:59:51.482281+0000","last_became_peered":"2024-03-26T17:59:51.482281+0000","last_unstale":"2024-03-26T17:59:52.726584+0000","last_undegraded":"2024-03-26T17:59:52.726584+0000","last_fullsized":"2024-03-26T17:59:52.726584+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:05:25.322809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724907+0000","last_change":"2024-03-26T17:59:46.440480+0000","last_active":"2024-03-26T17:59:52.724907+0000","last_peered":"2024-03-26T17:59:52.724907+0000","last_clean":"2024-03-26T17:59:52.724907+0000","last_became_active":"2024-03-26T17:59:46.439574+0000","last_became_peered":"2024-03-26T17:59:46.439574+0000","last_unstale":"2024-03-26T17:59:52.724907+0000","last_undegraded":"2024-03-26T17:59:52.724907+0000","last_fullsized":"2024-03-26T17:59:52.724907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:50:42.296440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461473+0000","last_change":"2024-03-26T17:59:47.229944+0000","last_active":"2024-03-26T17:59:52.461473+0000","last_peered":"2024-03-26T17:59:52.461473+0000","last_clean":"2024-03-26T17:59:52.461473+0000","last_became_active":"2024-03-26T17:59:47.229626+0000","last_became_peered":"2024-03-26T17:59:47.229626+0000","last_unstale":"2024-03-26T17:59:52.461473+0000","last_undegraded":"2024-03-26T17:59:52.461473+0000","last_fullsized":"2024-03-26T17:59:52.461473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:18.998051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724804+0000","last_change":"2024-03-26T17:59:51.486939+0000","last_active":"2024-03-26T17:59:52.724804+0000","last_peered":"2024-03-26T17:59:52.724804+0000","last_clean":"2024-03-26T17:59:52.724804+0000","last_became_active":"2024-03-26T17:59:51.486458+0000","last_became_peered":"2024-03-26T17:59:51.486458+0000","last_unstale":"2024-03-26T17:59:52.724804+0000","last_undegraded":"2024-03-26T17:59:52.724804+0000","last_fullsized":"2024-03-26T17:59:52.724804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:30:25.297900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725825+0000","last_change":"2024-03-26T17:59:46.435999+0000","last_active":"2024-03-26T17:59:52.725825+0000","last_peered":"2024-03-26T17:59:52.725825+0000","last_clean":"2024-03-26T17:59:52.725825+0000","last_became_active":"2024-03-26T17:59:46.435611+0000","last_became_peered":"2024-03-26T17:59:46.435611+0000","last_unstale":"2024-03-26T17:59:52.725825+0000","last_undegraded":"2024-03-26T17:59:52.725825+0000","last_fullsized":"2024-03-26T17:59:52.725825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:13:14.904792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462006+0000","last_change":"2024-03-26T17:59:47.236395+0000","last_active":"2024-03-26T17:59:52.462006+0000","last_peered":"2024-03-26T17:59:52.462006+0000","last_clean":"2024-03-26T17:59:52.462006+0000","last_became_active":"2024-03-26T17:59:47.236055+0000","last_became_peered":"2024-03-26T17:59:47.236055+0000","last_unstale":"2024-03-26T17:59:52.462006+0000","last_undegraded":"2024-03-26T17:59:52.462006+0000","last_fullsized":"2024-03-26T17:59:52.462006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:06:26.131419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725252+0000","last_change":"2024-03-26T17:59:51.484584+0000","last_active":"2024-03-26T17:59:52.725252+0000","last_peered":"2024-03-26T17:59:52.725252+0000","last_clean":"2024-03-26T17:59:52.725252+0000","last_became_active":"2024-03-26T17:59:51.484138+0000","last_became_peered":"2024-03-26T17:59:51.484138+0000","last_unstale":"2024-03-26T17:59:52.725252+0000","last_undegraded":"2024-03-26T17:59:52.725252+0000","last_fullsized":"2024-03-26T17:59:52.725252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:49:40.163174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.518214+0000","last_change":"2024-03-26T17:59:46.433243+0000","last_active":"2024-03-26T18:00:06.518214+0000","last_peered":"2024-03-26T18:00:06.518214+0000","last_clean":"2024-03-26T18:00:06.518214+0000","last_became_active":"2024-03-26T17:59:46.432939+0000","last_became_peered":"2024-03-26T17:59:46.432939+0000","last_unstale":"2024-03-26T18:00:06.518214+0000","last_undegraded":"2024-03-26T18:00:06.518214+0000","last_fullsized":"2024-03-26T18:00:06.518214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:44:09.473862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461770+0000","last_change":"2024-03-26T17:59:47.228882+0000","last_active":"2024-03-26T17:59:52.461770+0000","last_peered":"2024-03-26T17:59:52.461770+0000","last_clean":"2024-03-26T17:59:52.461770+0000","last_became_active":"2024-03-26T17:59:47.228601+0000","last_became_peered":"2024-03-26T17:59:47.228601+0000","last_unstale":"2024-03-26T17:59:52.461770+0000","last_undegraded":"2024-03-26T17:59:52.461770+0000","last_fullsized":"2024-03-26T17:59:52.461770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:07:46.835582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462024+0000","last_change":"2024-03-26T17:59:51.488010+0000","last_active":"2024-03-26T17:59:52.462024+0000","last_peered":"2024-03-26T17:59:52.462024+0000","last_clean":"2024-03-26T17:59:52.462024+0000","last_became_active":"2024-03-26T17:59:51.486406+0000","last_became_peered":"2024-03-26T17:59:51.486406+0000","last_unstale":"2024-03-26T17:59:52.462024+0000","last_undegraded":"2024-03-26T17:59:52.462024+0000","last_fullsized":"2024-03-26T17:59:52.462024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:54:49.102720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.521363+0000","last_change":"2024-03-26T17:59:46.438764+0000","last_active":"2024-03-26T18:00:06.521363+0000","last_peered":"2024-03-26T18:00:06.521363+0000","last_clean":"2024-03-26T18:00:06.521363+0000","last_became_active":"2024-03-26T17:59:46.438329+0000","last_became_peered":"2024-03-26T17:59:46.438329+0000","last_unstale":"2024-03-26T18:00:06.521363+0000","last_undegraded":"2024-03-26T18:00:06.521363+0000","last_fullsized":"2024-03-26T18:00:06.521363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:33:19.025134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725776+0000","last_change":"2024-03-26T17:59:47.237020+0000","last_active":"2024-03-26T17:59:52.725776+0000","last_peered":"2024-03-26T17:59:52.725776+0000","last_clean":"2024-03-26T17:59:52.725776+0000","last_became_active":"2024-03-26T17:59:47.236693+0000","last_became_peered":"2024-03-26T17:59:47.236693+0000","last_unstale":"2024-03-26T17:59:52.725776+0000","last_undegraded":"2024-03-26T17:59:52.725776+0000","last_fullsized":"2024-03-26T17:59:52.725776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:40:25.473994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725693+0000","last_change":"2024-03-26T17:59:51.482799+0000","last_active":"2024-03-26T17:59:52.725693+0000","last_peered":"2024-03-26T17:59:52.725693+0000","last_clean":"2024-03-26T17:59:52.725693+0000","last_became_active":"2024-03-26T17:59:51.482246+0000","last_became_peered":"2024-03-26T17:59:51.482246+0000","last_unstale":"2024-03-26T17:59:52.725693+0000","last_undegraded":"2024-03-26T17:59:52.725693+0000","last_fullsized":"2024-03-26T17:59:52.725693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:43:11.132952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461730+0000","last_change":"2024-03-26T17:59:46.438038+0000","last_active":"2024-03-26T17:59:52.461730+0000","last_peered":"2024-03-26T17:59:52.461730+0000","last_clean":"2024-03-26T17:59:52.461730+0000","last_became_active":"2024-03-26T17:59:46.437654+0000","last_became_peered":"2024-03-26T17:59:46.437654+0000","last_unstale":"2024-03-26T17:59:52.461730+0000","last_undegraded":"2024-03-26T17:59:52.461730+0000","last_fullsized":"2024-03-26T17:59:52.461730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:33:04.341028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724919+0000","last_change":"2024-03-26T17:59:47.235516+0000","last_active":"2024-03-26T17:59:52.724919+0000","last_peered":"2024-03-26T17:59:52.724919+0000","last_clean":"2024-03-26T17:59:52.724919+0000","last_became_active":"2024-03-26T17:59:47.235182+0000","last_became_peered":"2024-03-26T17:59:47.235182+0000","last_unstale":"2024-03-26T17:59:52.724919+0000","last_undegraded":"2024-03-26T17:59:52.724919+0000","last_fullsized":"2024-03-26T17:59:52.724919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:28:53.365700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461805+0000","last_change":"2024-03-26T17:59:51.477057+0000","last_active":"2024-03-26T17:59:52.461805+0000","last_peered":"2024-03-26T17:59:52.461805+0000","last_clean":"2024-03-26T17:59:52.461805+0000","last_became_active":"2024-03-26T17:59:51.476576+0000","last_became_peered":"2024-03-26T17:59:51.476576+0000","last_unstale":"2024-03-26T17:59:52.461805+0000","last_undegraded":"2024-03-26T17:59:52.461805+0000","last_fullsized":"2024-03-26T17:59:52.461805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:42.533519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725178+0000","last_change":"2024-03-26T17:59:46.437494+0000","last_active":"2024-03-26T17:59:52.725178+0000","last_peered":"2024-03-26T17:59:52.725178+0000","last_clean":"2024-03-26T17:59:52.725178+0000","last_became_active":"2024-03-26T17:59:46.437184+0000","last_became_peered":"2024-03-26T17:59:46.437184+0000","last_unstale":"2024-03-26T17:59:52.725178+0000","last_undegraded":"2024-03-26T17:59:52.725178+0000","last_fullsized":"2024-03-26T17:59:52.725178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:28:21.560104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.669230+0000","last_change":"2024-03-26T17:59:47.232436+0000","last_active":"2024-03-26T17:59:52.669230+0000","last_peered":"2024-03-26T17:59:52.669230+0000","last_clean":"2024-03-26T17:59:52.669230+0000","last_became_active":"2024-03-26T17:59:47.232129+0000","last_became_peered":"2024-03-26T17:59:47.232129+0000","last_unstale":"2024-03-26T17:59:52.669230+0000","last_undegraded":"2024-03-26T17:59:52.669230+0000","last_fullsized":"2024-03-26T17:59:52.669230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:27:54.504624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461845+0000","last_change":"2024-03-26T17:59:51.475972+0000","last_active":"2024-03-26T17:59:52.461845+0000","last_peered":"2024-03-26T17:59:52.461845+0000","last_clean":"2024-03-26T17:59:52.461845+0000","last_became_active":"2024-03-26T17:59:51.475531+0000","last_became_peered":"2024-03-26T17:59:51.475531+0000","last_unstale":"2024-03-26T17:59:52.461845+0000","last_undegraded":"2024-03-26T17:59:52.461845+0000","last_fullsized":"2024-03-26T17:59:52.461845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:43:23.639809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461902+0000","last_change":"2024-03-26T17:59:46.432980+0000","last_active":"2024-03-26T17:59:52.461902+0000","last_peered":"2024-03-26T17:59:52.461902+0000","last_clean":"2024-03-26T17:59:52.461902+0000","last_became_active":"2024-03-26T17:59:46.432095+0000","last_became_peered":"2024-03-26T17:59:46.432095+0000","last_unstale":"2024-03-26T17:59:52.461902+0000","last_undegraded":"2024-03-26T17:59:52.461902+0000","last_fullsized":"2024-03-26T17:59:52.461902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:19:36.897555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.669072+0000","last_change":"2024-03-26T17:59:47.229623+0000","last_active":"2024-03-26T17:59:52.669072+0000","last_peered":"2024-03-26T17:59:52.669072+0000","last_clean":"2024-03-26T17:59:52.669072+0000","last_became_active":"2024-03-26T17:59:47.228781+0000","last_became_peered":"2024-03-26T17:59:47.228781+0000","last_unstale":"2024-03-26T17:59:52.669072+0000","last_undegraded":"2024-03-26T17:59:52.669072+0000","last_fullsized":"2024-03-26T17:59:52.669072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:54:22.171273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461999+0000","last_change":"2024-03-26T17:59:51.483247+0000","last_active":"2024-03-26T17:59:52.461999+0000","last_peered":"2024-03-26T17:59:52.461999+0000","last_clean":"2024-03-26T17:59:52.461999+0000","last_became_active":"2024-03-26T17:59:51.482854+0000","last_became_peered":"2024-03-26T17:59:51.482854+0000","last_unstale":"2024-03-26T17:59:52.461999+0000","last_undegraded":"2024-03-26T17:59:52.461999+0000","last_fullsized":"2024-03-26T17:59:52.461999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:24:13.532986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726549+0000","last_change":"2024-03-26T17:59:47.238866+0000","last_active":"2024-03-26T17:59:52.726549+0000","last_peered":"2024-03-26T17:59:52.726549+0000","last_clean":"2024-03-26T17:59:52.726549+0000","last_became_active":"2024-03-26T17:59:47.238366+0000","last_became_peered":"2024-03-26T17:59:47.238366+0000","last_unstale":"2024-03-26T17:59:52.726549+0000","last_undegraded":"2024-03-26T17:59:52.726549+0000","last_fullsized":"2024-03-26T17:59:52.726549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:04.045820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.522403+0000","last_change":"2024-03-26T17:59:46.433046+0000","last_active":"2024-03-26T18:00:06.522403+0000","last_peered":"2024-03-26T18:00:06.522403+0000","last_clean":"2024-03-26T18:00:06.522403+0000","last_became_active":"2024-03-26T17:59:46.432585+0000","last_became_peered":"2024-03-26T17:59:46.432585+0000","last_unstale":"2024-03-26T18:00:06.522403+0000","last_undegraded":"2024-03-26T18:00:06.522403+0000","last_fullsized":"2024-03-26T18:00:06.522403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:17.051521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462236+0000","last_change":"2024-03-26T17:59:51.478866+0000","last_active":"2024-03-26T17:59:52.462236+0000","last_peered":"2024-03-26T17:59:52.462236+0000","last_clean":"2024-03-26T17:59:52.462236+0000","last_became_active":"2024-03-26T17:59:51.478497+0000","last_became_peered":"2024-03-26T17:59:51.478497+0000","last_unstale":"2024-03-26T17:59:52.462236+0000","last_undegraded":"2024-03-26T17:59:52.462236+0000","last_fullsized":"2024-03-26T17:59:52.462236+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:15:05.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725181+0000","last_change":"2024-03-26T17:59:47.232349+0000","last_active":"2024-03-26T17:59:52.725181+0000","last_peered":"2024-03-26T17:59:52.725181+0000","last_clean":"2024-03-26T17:59:52.725181+0000","last_became_active":"2024-03-26T17:59:47.232001+0000","last_became_peered":"2024-03-26T17:59:47.232001+0000","last_unstale":"2024-03-26T17:59:52.725181+0000","last_undegraded":"2024-03-26T17:59:52.725181+0000","last_fullsized":"2024-03-26T17:59:52.725181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:42:14.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.513016+0000","last_change":"2024-03-26T17:59:46.431522+0000","last_active":"2024-03-26T18:00:05.513016+0000","last_peered":"2024-03-26T18:00:05.513016+0000","last_clean":"2024-03-26T18:00:05.513016+0000","last_became_active":"2024-03-26T17:59:46.431201+0000","last_became_peered":"2024-03-26T17:59:46.431201+0000","last_unstale":"2024-03-26T18:00:05.513016+0000","last_undegraded":"2024-03-26T18:00:05.513016+0000","last_fullsized":"2024-03-26T18:00:05.513016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:11.298111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461649+0000","last_change":"2024-03-26T17:58:38.167621+0000","last_active":"2024-03-26T17:59:52.461649+0000","last_peered":"2024-03-26T17:59:52.461649+0000","last_clean":"2024-03-26T17:59:52.461649+0000","last_became_active":"2024-03-26T17:58:38.167247+0000","last_became_peered":"2024-03-26T17:58:38.167247+0000","last_unstale":"2024-03-26T17:59:52.461649+0000","last_undegraded":"2024-03-26T17:59:52.461649+0000","last_fullsized":"2024-03-26T17:59:52.461649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:55:11.216723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:55:11.216723+0000","last_clean_scrub_stamp":"2024-03-26T17:55:11.216723+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:59:04.140225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726085+0000","last_change":"2024-03-26T17:59:51.480354+0000","last_active":"2024-03-26T17:59:52.726085+0000","last_peered":"2024-03-26T17:59:52.726085+0000","last_clean":"2024-03-26T17:59:52.726085+0000","last_became_active":"2024-03-26T17:59:51.479474+0000","last_became_peered":"2024-03-26T17:59:51.479474+0000","last_unstale":"2024-03-26T17:59:52.726085+0000","last_undegraded":"2024-03-26T17:59:52.726085+0000","last_fullsized":"2024-03-26T17:59:52.726085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:48:31.846364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461227+0000","last_change":"2024-03-26T17:59:47.226036+0000","last_active":"2024-03-26T17:59:52.461227+0000","last_peered":"2024-03-26T17:59:52.461227+0000","last_clean":"2024-03-26T17:59:52.461227+0000","last_became_active":"2024-03-26T17:59:47.225722+0000","last_became_peered":"2024-03-26T17:59:47.225722+0000","last_unstale":"2024-03-26T17:59:52.461227+0000","last_undegraded":"2024-03-26T17:59:52.461227+0000","last_fullsized":"2024-03-26T17:59:52.461227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:28:39.310849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:06.518760+0000","last_change":"2024-03-26T17:59:46.432635+0000","last_active":"2024-03-26T18:00:06.518760+0000","last_peered":"2024-03-26T18:00:06.518760+0000","last_clean":"2024-03-26T18:00:06.518760+0000","last_became_active":"2024-03-26T17:59:46.432317+0000","last_became_peered":"2024-03-26T17:59:46.432317+0000","last_unstale":"2024-03-26T18:00:06.518760+0000","last_undegraded":"2024-03-26T18:00:06.518760+0000","last_fullsized":"2024-03-26T18:00:06.518760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:48:15.018906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725253+0000","last_change":"2024-03-26T17:59:51.480678+0000","last_active":"2024-03-26T17:59:52.725253+0000","last_peered":"2024-03-26T17:59:52.725253+0000","last_clean":"2024-03-26T17:59:52.725253+0000","last_became_active":"2024-03-26T17:59:51.480290+0000","last_became_peered":"2024-03-26T17:59:51.480290+0000","last_unstale":"2024-03-26T17:59:52.725253+0000","last_undegraded":"2024-03-26T17:59:52.725253+0000","last_fullsized":"2024-03-26T17:59:52.725253+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:02:20.279963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461932+0000","last_change":"2024-03-26T17:59:46.445569+0000","last_active":"2024-03-26T17:59:52.461932+0000","last_peered":"2024-03-26T17:59:52.461932+0000","last_clean":"2024-03-26T17:59:52.461932+0000","last_became_active":"2024-03-26T17:59:46.445114+0000","last_became_peered":"2024-03-26T17:59:46.445114+0000","last_unstale":"2024-03-26T17:59:52.461932+0000","last_undegraded":"2024-03-26T17:59:52.461932+0000","last_fullsized":"2024-03-26T17:59:52.461932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:18:59.367401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461911+0000","last_change":"2024-03-26T17:59:47.227901+0000","last_active":"2024-03-26T17:59:52.461911+0000","last_peered":"2024-03-26T17:59:52.461911+0000","last_clean":"2024-03-26T17:59:52.461911+0000","last_became_active":"2024-03-26T17:59:47.227336+0000","last_became_peered":"2024-03-26T17:59:47.227336+0000","last_unstale":"2024-03-26T17:59:52.461911+0000","last_undegraded":"2024-03-26T17:59:52.461911+0000","last_fullsized":"2024-03-26T17:59:52.461911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:44:25.326477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724557+0000","last_change":"2024-03-26T17:59:51.482051+0000","last_active":"2024-03-26T17:59:52.724557+0000","last_peered":"2024-03-26T17:59:52.724557+0000","last_clean":"2024-03-26T17:59:52.724557+0000","last_became_active":"2024-03-26T17:59:51.481417+0000","last_became_peered":"2024-03-26T17:59:51.481417+0000","last_unstale":"2024-03-26T17:59:52.724557+0000","last_undegraded":"2024-03-26T17:59:52.724557+0000","last_fullsized":"2024-03-26T17:59:52.724557+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:41:33.294253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725444+0000","last_change":"2024-03-26T17:59:46.436832+0000","last_active":"2024-03-26T17:59:52.725444+0000","last_peered":"2024-03-26T17:59:52.725444+0000","last_clean":"2024-03-26T17:59:52.725444+0000","last_became_active":"2024-03-26T17:59:46.433953+0000","last_became_peered":"2024-03-26T17:59:46.433953+0000","last_unstale":"2024-03-26T17:59:52.725444+0000","last_undegraded":"2024-03-26T17:59:52.725444+0000","last_fullsized":"2024-03-26T17:59:52.725444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:18:30.282277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725997+0000","last_change":"2024-03-26T17:59:47.241212+0000","last_active":"2024-03-26T17:59:52.725997+0000","last_peered":"2024-03-26T17:59:52.725997+0000","last_clean":"2024-03-26T17:59:52.725997+0000","last_became_active":"2024-03-26T17:59:47.240850+0000","last_became_peered":"2024-03-26T17:59:47.240850+0000","last_unstale":"2024-03-26T17:59:52.725997+0000","last_undegraded":"2024-03-26T17:59:52.725997+0000","last_fullsized":"2024-03-26T17:59:52.725997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:25:15.777349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725633+0000","last_change":"2024-03-26T17:59:51.474603+0000","last_active":"2024-03-26T17:59:52.725633+0000","last_peered":"2024-03-26T17:59:52.725633+0000","last_clean":"2024-03-26T17:59:52.725633+0000","last_became_active":"2024-03-26T17:59:51.474069+0000","last_became_peered":"2024-03-26T17:59:51.474069+0000","last_unstale":"2024-03-26T17:59:52.725633+0000","last_undegraded":"2024-03-26T17:59:52.725633+0000","last_fullsized":"2024-03-26T17:59:52.725633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:32:05.459266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.511203+0000","last_change":"2024-03-26T17:59:46.435318+0000","last_active":"2024-03-26T18:00:05.511203+0000","last_peered":"2024-03-26T18:00:05.511203+0000","last_clean":"2024-03-26T18:00:05.511203+0000","last_became_active":"2024-03-26T17:59:46.434982+0000","last_became_peered":"2024-03-26T17:59:46.434982+0000","last_unstale":"2024-03-26T18:00:05.511203+0000","last_undegraded":"2024-03-26T18:00:05.511203+0000","last_fullsized":"2024-03-26T18:00:05.511203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:24:28.364973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726407+0000","last_change":"2024-03-26T17:59:47.239552+0000","last_active":"2024-03-26T17:59:52.726407+0000","last_peered":"2024-03-26T17:59:52.726407+0000","last_clean":"2024-03-26T17:59:52.726407+0000","last_became_active":"2024-03-26T17:59:47.238027+0000","last_became_peered":"2024-03-26T17:59:47.238027+0000","last_unstale":"2024-03-26T17:59:52.726407+0000","last_undegraded":"2024-03-26T17:59:52.726407+0000","last_fullsized":"2024-03-26T17:59:52.726407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:49:39.158517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724351+0000","last_change":"2024-03-26T17:59:51.486887+0000","last_active":"2024-03-26T17:59:52.724351+0000","last_peered":"2024-03-26T17:59:52.724351+0000","last_clean":"2024-03-26T17:59:52.724351+0000","last_became_active":"2024-03-26T17:59:51.486426+0000","last_became_peered":"2024-03-26T17:59:51.486426+0000","last_unstale":"2024-03-26T17:59:52.724351+0000","last_undegraded":"2024-03-26T17:59:52.724351+0000","last_fullsized":"2024-03-26T17:59:52.724351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:56.561561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.512858+0000","last_change":"2024-03-26T17:59:46.437858+0000","last_active":"2024-03-26T18:00:05.512858+0000","last_peered":"2024-03-26T18:00:05.512858+0000","last_clean":"2024-03-26T18:00:05.512858+0000","last_became_active":"2024-03-26T17:59:46.437499+0000","last_became_peered":"2024-03-26T17:59:46.437499+0000","last_unstale":"2024-03-26T18:00:05.512858+0000","last_undegraded":"2024-03-26T18:00:05.512858+0000","last_fullsized":"2024-03-26T18:00:05.512858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:42:01.913000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461449+0000","last_change":"2024-03-26T17:59:47.221959+0000","last_active":"2024-03-26T17:59:52.461449+0000","last_peered":"2024-03-26T17:59:52.461449+0000","last_clean":"2024-03-26T17:59:52.461449+0000","last_became_active":"2024-03-26T17:59:47.221550+0000","last_became_peered":"2024-03-26T17:59:47.221550+0000","last_unstale":"2024-03-26T17:59:52.461449+0000","last_undegraded":"2024-03-26T17:59:52.461449+0000","last_fullsized":"2024-03-26T17:59:52.461449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:53:21.354652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725733+0000","last_change":"2024-03-26T17:59:51.487487+0000","last_active":"2024-03-26T17:59:52.725733+0000","last_peered":"2024-03-26T17:59:52.725733+0000","last_clean":"2024-03-26T17:59:52.725733+0000","last_became_active":"2024-03-26T17:59:51.484494+0000","last_became_peered":"2024-03-26T17:59:51.484494+0000","last_unstale":"2024-03-26T17:59:52.725733+0000","last_undegraded":"2024-03-26T17:59:52.725733+0000","last_fullsized":"2024-03-26T17:59:52.725733+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:29.849739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.512837+0000","last_change":"2024-03-26T17:59:46.437507+0000","last_active":"2024-03-26T18:00:05.512837+0000","last_peered":"2024-03-26T18:00:05.512837+0000","last_clean":"2024-03-26T18:00:05.512837+0000","last_became_active":"2024-03-26T17:59:46.437146+0000","last_became_peered":"2024-03-26T17:59:46.437146+0000","last_unstale":"2024-03-26T18:00:05.512837+0000","last_undegraded":"2024-03-26T18:00:05.512837+0000","last_fullsized":"2024-03-26T18:00:05.512837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:03:44.924632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724636+0000","last_change":"2024-03-26T17:59:47.227678+0000","last_active":"2024-03-26T17:59:52.724636+0000","last_peered":"2024-03-26T17:59:52.724636+0000","last_clean":"2024-03-26T17:59:52.724636+0000","last_became_active":"2024-03-26T17:59:47.224858+0000","last_became_peered":"2024-03-26T17:59:47.224858+0000","last_unstale":"2024-03-26T17:59:52.724636+0000","last_undegraded":"2024-03-26T17:59:52.724636+0000","last_fullsized":"2024-03-26T17:59:52.724636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:39.404469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724491+0000","last_change":"2024-03-26T17:59:51.487800+0000","last_active":"2024-03-26T17:59:52.724491+0000","last_peered":"2024-03-26T17:59:52.724491+0000","last_clean":"2024-03-26T17:59:52.724491+0000","last_became_active":"2024-03-26T17:59:51.486933+0000","last_became_peered":"2024-03-26T17:59:51.486933+0000","last_unstale":"2024-03-26T17:59:52.724491+0000","last_undegraded":"2024-03-26T17:59:52.724491+0000","last_fullsized":"2024-03-26T17:59:52.724491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:05:03.333414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724688+0000","last_change":"2024-03-26T17:59:46.446331+0000","last_active":"2024-03-26T17:59:52.724688+0000","last_peered":"2024-03-26T17:59:52.724688+0000","last_clean":"2024-03-26T17:59:52.724688+0000","last_became_active":"2024-03-26T17:59:46.446013+0000","last_became_peered":"2024-03-26T17:59:46.446013+0000","last_unstale":"2024-03-26T17:59:52.724688+0000","last_undegraded":"2024-03-26T17:59:52.724688+0000","last_fullsized":"2024-03-26T17:59:52.724688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:59:40.458684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725709+0000","last_change":"2024-03-26T17:59:47.236616+0000","last_active":"2024-03-26T17:59:52.725709+0000","last_peered":"2024-03-26T17:59:52.725709+0000","last_clean":"2024-03-26T17:59:52.725709+0000","last_became_active":"2024-03-26T17:59:47.236261+0000","last_became_peered":"2024-03-26T17:59:47.236261+0000","last_unstale":"2024-03-26T17:59:52.725709+0000","last_undegraded":"2024-03-26T17:59:52.725709+0000","last_fullsized":"2024-03-26T17:59:52.725709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:54:45.319703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724401+0000","last_change":"2024-03-26T17:59:51.482500+0000","last_active":"2024-03-26T17:59:52.724401+0000","last_peered":"2024-03-26T17:59:52.724401+0000","last_clean":"2024-03-26T17:59:52.724401+0000","last_became_active":"2024-03-26T17:59:51.482055+0000","last_became_peered":"2024-03-26T17:59:51.482055+0000","last_unstale":"2024-03-26T17:59:52.724401+0000","last_undegraded":"2024-03-26T17:59:52.724401+0000","last_fullsized":"2024-03-26T17:59:52.724401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:27:10.278683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T18:00:05.512933+0000","last_change":"2024-03-26T17:59:46.438600+0000","last_active":"2024-03-26T18:00:05.512933+0000","last_peered":"2024-03-26T18:00:05.512933+0000","last_clean":"2024-03-26T18:00:05.512933+0000","last_became_active":"2024-03-26T17:59:46.438272+0000","last_became_peered":"2024-03-26T17:59:46.438272+0000","last_unstale":"2024-03-26T18:00:05.512933+0000","last_undegraded":"2024-03-26T18:00:05.512933+0000","last_fullsized":"2024-03-26T18:00:05.512933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:18:00.551008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461932+0000","last_change":"2024-03-26T17:59:47.227500+0000","last_active":"2024-03-26T17:59:52.461932+0000","last_peered":"2024-03-26T17:59:52.461932+0000","last_clean":"2024-03-26T17:59:52.461932+0000","last_became_active":"2024-03-26T17:59:47.227184+0000","last_became_peered":"2024-03-26T17:59:47.227184+0000","last_unstale":"2024-03-26T17:59:52.461932+0000","last_undegraded":"2024-03-26T17:59:52.461932+0000","last_fullsized":"2024-03-26T17:59:52.461932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:41:22.876846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724125+0000","last_change":"2024-03-26T17:59:51.479724+0000","last_active":"2024-03-26T17:59:52.724125+0000","last_peered":"2024-03-26T17:59:52.724125+0000","last_clean":"2024-03-26T17:59:52.724125+0000","last_became_active":"2024-03-26T17:59:51.479374+0000","last_became_peered":"2024-03-26T17:59:51.479374+0000","last_unstale":"2024-03-26T17:59:52.724125+0000","last_undegraded":"2024-03-26T17:59:52.724125+0000","last_fullsized":"2024-03-26T17:59:52.724125+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:57:23.213546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725525+0000","last_change":"2024-03-26T17:59:46.442609+0000","last_active":"2024-03-26T17:59:52.725525+0000","last_peered":"2024-03-26T17:59:52.725525+0000","last_clean":"2024-03-26T17:59:52.725525+0000","last_became_active":"2024-03-26T17:59:46.442311+0000","last_became_peered":"2024-03-26T17:59:46.442311+0000","last_unstale":"2024-03-26T17:59:52.725525+0000","last_undegraded":"2024-03-26T17:59:52.725525+0000","last_fullsized":"2024-03-26T17:59:52.725525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:02:40.141449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462370+0000","last_change":"2024-03-26T17:59:47.228152+0000","last_active":"2024-03-26T17:59:52.462370+0000","last_peered":"2024-03-26T17:59:52.462370+0000","last_clean":"2024-03-26T17:59:52.462370+0000","last_became_active":"2024-03-26T17:59:47.227859+0000","last_became_peered":"2024-03-26T17:59:47.227859+0000","last_unstale":"2024-03-26T17:59:52.462370+0000","last_undegraded":"2024-03-26T17:59:52.462370+0000","last_fullsized":"2024-03-26T17:59:52.462370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:16:59.445039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724959+0000","last_change":"2024-03-26T17:59:51.487576+0000","last_active":"2024-03-26T17:59:52.724959+0000","last_peered":"2024-03-26T17:59:52.724959+0000","last_clean":"2024-03-26T17:59:52.724959+0000","last_became_active":"2024-03-26T17:59:51.486995+0000","last_became_peered":"2024-03-26T17:59:51.486995+0000","last_unstale":"2024-03-26T17:59:52.724959+0000","last_undegraded":"2024-03-26T17:59:52.724959+0000","last_fullsized":"2024-03-26T17:59:52.724959+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:58.061260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726185+0000","last_change":"2024-03-26T17:59:46.434433+0000","last_active":"2024-03-26T17:59:52.726185+0000","last_peered":"2024-03-26T17:59:52.726185+0000","last_clean":"2024-03-26T17:59:52.726185+0000","last_became_active":"2024-03-26T17:59:46.434117+0000","last_became_peered":"2024-03-26T17:59:46.434117+0000","last_unstale":"2024-03-26T17:59:52.726185+0000","last_undegraded":"2024-03-26T17:59:52.726185+0000","last_fullsized":"2024-03-26T17:59:52.726185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:28.083880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462067+0000","last_change":"2024-03-26T17:59:47.232406+0000","last_active":"2024-03-26T17:59:52.462067+0000","last_peered":"2024-03-26T17:59:52.462067+0000","last_clean":"2024-03-26T17:59:52.462067+0000","last_became_active":"2024-03-26T17:59:47.232076+0000","last_became_peered":"2024-03-26T17:59:47.232076+0000","last_unstale":"2024-03-26T17:59:52.462067+0000","last_undegraded":"2024-03-26T17:59:52.462067+0000","last_fullsized":"2024-03-26T17:59:52.462067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:46:23.159106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724630+0000","last_change":"2024-03-26T17:59:51.475555+0000","last_active":"2024-03-26T17:59:52.724630+0000","last_peered":"2024-03-26T17:59:52.724630+0000","last_clean":"2024-03-26T17:59:52.724630+0000","last_became_active":"2024-03-26T17:59:51.475136+0000","last_became_peered":"2024-03-26T17:59:51.475136+0000","last_unstale":"2024-03-26T17:59:52.724630+0000","last_undegraded":"2024-03-26T17:59:52.724630+0000","last_fullsized":"2024-03-26T17:59:52.724630+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:28:07.279237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461907+0000","last_change":"2024-03-26T17:59:46.437510+0000","last_active":"2024-03-26T17:59:52.461907+0000","last_peered":"2024-03-26T17:59:52.461907+0000","last_clean":"2024-03-26T17:59:52.461907+0000","last_became_active":"2024-03-26T17:59:46.437191+0000","last_became_peered":"2024-03-26T17:59:46.437191+0000","last_unstale":"2024-03-26T17:59:52.461907+0000","last_undegraded":"2024-03-26T17:59:52.461907+0000","last_fullsized":"2024-03-26T17:59:52.461907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:19:13.759085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.462315+0000","last_change":"2024-03-26T17:59:47.225692+0000","last_active":"2024-03-26T17:59:52.462315+0000","last_peered":"2024-03-26T17:59:52.462315+0000","last_clean":"2024-03-26T17:59:52.462315+0000","last_became_active":"2024-03-26T17:59:47.225363+0000","last_became_peered":"2024-03-26T17:59:47.225363+0000","last_unstale":"2024-03-26T17:59:52.462315+0000","last_undegraded":"2024-03-26T17:59:52.462315+0000","last_fullsized":"2024-03-26T17:59:52.462315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:01:44.318833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724774+0000","last_change":"2024-03-26T17:59:47.227155+0000","last_active":"2024-03-26T17:59:52.724774+0000","last_peered":"2024-03-26T17:59:52.724774+0000","last_clean":"2024-03-26T17:59:52.724774+0000","last_became_active":"2024-03-26T17:59:47.226504+0000","last_became_peered":"2024-03-26T17:59:47.226504+0000","last_unstale":"2024-03-26T17:59:52.724774+0000","last_undegraded":"2024-03-26T17:59:52.724774+0000","last_fullsized":"2024-03-26T17:59:52.724774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:16:48.427819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726000+0000","last_change":"2024-03-26T17:59:47.230394+0000","last_active":"2024-03-26T17:59:52.726000+0000","last_peered":"2024-03-26T17:59:52.726000+0000","last_clean":"2024-03-26T17:59:52.726000+0000","last_became_active":"2024-03-26T17:59:47.228367+0000","last_became_peered":"2024-03-26T17:59:47.228367+0000","last_unstale":"2024-03-26T17:59:52.726000+0000","last_undegraded":"2024-03-26T17:59:52.726000+0000","last_fullsized":"2024-03-26T17:59:52.726000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:59:04.158938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724568+0000","last_change":"2024-03-26T17:59:47.230877+0000","last_active":"2024-03-26T17:59:52.724568+0000","last_peered":"2024-03-26T17:59:52.724568+0000","last_clean":"2024-03-26T17:59:52.724568+0000","last_became_active":"2024-03-26T17:59:47.230478+0000","last_became_peered":"2024-03-26T17:59:47.230478+0000","last_unstale":"2024-03-26T17:59:52.724568+0000","last_undegraded":"2024-03-26T17:59:52.724568+0000","last_fullsized":"2024-03-26T17:59:52.724568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:46:25.867718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725182+0000","last_change":"2024-03-26T17:59:47.237012+0000","last_active":"2024-03-26T17:59:52.725182+0000","last_peered":"2024-03-26T17:59:52.725182+0000","last_clean":"2024-03-26T17:59:52.725182+0000","last_became_active":"2024-03-26T17:59:47.236705+0000","last_became_peered":"2024-03-26T17:59:47.236705+0000","last_unstale":"2024-03-26T17:59:52.725182+0000","last_undegraded":"2024-03-26T17:59:52.725182+0000","last_fullsized":"2024-03-26T17:59:52.725182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:34:41.379731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461829+0000","last_change":"2024-03-26T17:59:47.227641+0000","last_active":"2024-03-26T17:59:52.461829+0000","last_peered":"2024-03-26T17:59:52.461829+0000","last_clean":"2024-03-26T17:59:52.461829+0000","last_became_active":"2024-03-26T17:59:47.226921+0000","last_became_peered":"2024-03-26T17:59:47.226921+0000","last_unstale":"2024-03-26T17:59:52.461829+0000","last_undegraded":"2024-03-26T17:59:52.461829+0000","last_fullsized":"2024-03-26T17:59:52.461829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:48.442562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726275+0000","last_change":"2024-03-26T17:59:47.232387+0000","last_active":"2024-03-26T17:59:52.726275+0000","last_peered":"2024-03-26T17:59:52.726275+0000","last_clean":"2024-03-26T17:59:52.726275+0000","last_became_active":"2024-03-26T17:59:47.231934+0000","last_became_peered":"2024-03-26T17:59:47.231934+0000","last_unstale":"2024-03-26T17:59:52.726275+0000","last_undegraded":"2024-03-26T17:59:52.726275+0000","last_fullsized":"2024-03-26T17:59:52.726275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:32.006136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725108+0000","last_change":"2024-03-26T17:59:47.226178+0000","last_active":"2024-03-26T17:59:52.725108+0000","last_peered":"2024-03-26T17:59:52.725108+0000","last_clean":"2024-03-26T17:59:52.725108+0000","last_became_active":"2024-03-26T17:59:47.224692+0000","last_became_peered":"2024-03-26T17:59:47.224692+0000","last_unstale":"2024-03-26T17:59:52.725108+0000","last_undegraded":"2024-03-26T17:59:52.725108+0000","last_fullsized":"2024-03-26T17:59:52.725108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:18.617841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726177+0000","last_change":"2024-03-26T17:59:47.238289+0000","last_active":"2024-03-26T17:59:52.726177+0000","last_peered":"2024-03-26T17:59:52.726177+0000","last_clean":"2024-03-26T17:59:52.726177+0000","last_became_active":"2024-03-26T17:59:47.237904+0000","last_became_peered":"2024-03-26T17:59:47.237904+0000","last_unstale":"2024-03-26T17:59:52.726177+0000","last_undegraded":"2024-03-26T17:59:52.726177+0000","last_fullsized":"2024-03-26T17:59:52.726177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:57:42.695335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.726249+0000","last_change":"2024-03-26T17:59:47.234780+0000","last_active":"2024-03-26T17:59:52.726249+0000","last_peered":"2024-03-26T17:59:52.726249+0000","last_clean":"2024-03-26T17:59:52.726249+0000","last_became_active":"2024-03-26T17:59:47.234478+0000","last_became_peered":"2024-03-26T17:59:47.234478+0000","last_unstale":"2024-03-26T17:59:52.726249+0000","last_undegraded":"2024-03-26T17:59:52.726249+0000","last_fullsized":"2024-03-26T17:59:52.726249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:57:43.675947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.725390+0000","last_change":"2024-03-26T17:59:47.240057+0000","last_active":"2024-03-26T17:59:52.725390+0000","last_peered":"2024-03-26T17:59:52.725390+0000","last_clean":"2024-03-26T17:59:52.725390+0000","last_became_active":"2024-03-26T17:59:47.239725+0000","last_became_peered":"2024-03-26T17:59:47.239725+0000","last_unstale":"2024-03-26T17:59:52.725390+0000","last_undegraded":"2024-03-26T17:59:52.725390+0000","last_fullsized":"2024-03-26T17:59:52.725390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:45.364403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724433+0000","last_change":"2024-03-26T17:59:47.228059+0000","last_active":"2024-03-26T17:59:52.724433+0000","last_peered":"2024-03-26T17:59:52.724433+0000","last_clean":"2024-03-26T17:59:52.724433+0000","last_became_active":"2024-03-26T17:59:47.227666+0000","last_became_peered":"2024-03-26T17:59:47.227666+0000","last_unstale":"2024-03-26T17:59:52.724433+0000","last_undegraded":"2024-03-26T17:59:52.724433+0000","last_fullsized":"2024-03-26T17:59:52.724433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:19:26.440271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.461645+0000","last_change":"2024-03-26T17:59:47.228705+0000","last_active":"2024-03-26T17:59:52.461645+0000","last_peered":"2024-03-26T17:59:52.461645+0000","last_clean":"2024-03-26T17:59:52.461645+0000","last_became_active":"2024-03-26T17:59:47.228425+0000","last_became_peered":"2024-03-26T17:59:47.228425+0000","last_unstale":"2024-03-26T17:59:52.461645+0000","last_undegraded":"2024-03-26T17:59:52.461645+0000","last_fullsized":"2024-03-26T17:59:52.461645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:43:32.055816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T17:59:52.724933+0000","last_change":"2024-03-26T17:59:47.232080+0000","last_active":"2024-03-26T17:59:52.724933+0000","last_peered":"2024-03-26T17:59:52.724933+0000","last_clean":"2024-03-26T17:59:52.724933+0000","last_became_active":"2024-03-26T17:59:47.231672+0000","last_became_peered":"2024-03-26T17:59:47.231672+0000","last_unstale":"2024-03-26T17:59:52.724933+0000","last_undegraded":"2024-03-26T17:59:52.724933+0000","last_fullsized":"2024-03-26T17:59:52.724933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:39.962065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250240,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5576,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":92,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":18238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":480266,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612629,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1074670,"data_compressed":50921,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":480848,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":480892,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1072426,"data_compressed":50921,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":484108,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":481744,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":484142,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149753,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":480460,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888116,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":480288,"data_compressed":14705,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":1070816,"data_compressed":50921,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34580,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34580,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T18:00:10.668 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-26T18:00:10.669 INFO:tasks.cephadm:Setting up client nodes... 2024-03-26T18:00:10.669 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-26T18:00:10.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:10 smithi171 ceph-mon[36033]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-03-26T18:00:10.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:10 smithi171 ceph-mon[36033]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:10.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:10 smithi171 ceph-mon[36033]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:10.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:10 smithi171 ceph-mon[36033]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T18:00:10.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:10 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/479878660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:00:10.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:10 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2080088516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:10.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:10 smithi115 ceph-mon[36174]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-03-26T18:00:10.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:10 smithi115 ceph-mon[36174]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:10.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:10 smithi115 ceph-mon[36174]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:00:10.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:10 smithi115 ceph-mon[36174]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T18:00:10.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:10 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/479878660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:00:10.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:10 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2080088516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:11.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:11 smithi049 ceph-mon[32331]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:00:11.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:11 smithi171 ceph-mon[36033]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:00:11.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:11 smithi115 ceph-mon[36174]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:00:12.677 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:12 smithi049 ceph-mon[32331]: from='client.24827 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:00:12.678 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:12 smithi049 ceph-mon[32331]: pgmap v304: 129 pgs: 129 active+clean; 621 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.0 KiB/s wr, 13 op/s 2024-03-26T18:00:12.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.809+0000 7fc7b9011700 1 -- 172.21.15.49:0/1314639045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4101300 msgr2=0x7fc7b4101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:12.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.809+0000 7fc7b9011700 1 --2- 172.21.15.49:0/1314639045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4101300 0x7fc7b4101720 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40099c0 tx=0x7fc7a4009cd0 comp rx=0 tx=0).stop 2024-03-26T18:00:12.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.809+0000 7fc7b9011700 1 -- 172.21.15.49:0/1314639045 shutdown_connections 2024-03-26T18:00:12.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.809+0000 7fc7b9011700 1 --2- 172.21.15.49:0/1314639045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4102ea0 0x7fc7b4109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:12.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.809+0000 7fc7b9011700 1 --2- 172.21.15.49:0/1314639045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4101300 0x7fc7b4101720 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:12.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.809+0000 7fc7b9011700 1 --2- 172.21.15.49:0/1314639045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc7b4102500 0x7fc7b4102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:12.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.809+0000 7fc7b9011700 1 -- 172.21.15.49:0/1314639045 >> 172.21.15.49:0/1314639045 conn(0x7fc7b40fc8a0 msgr2=0x7fc7b40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:12.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.809+0000 7fc7b9011700 1 -- 172.21.15.49:0/1314639045 shutdown_connections 2024-03-26T18:00:12.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.809+0000 7fc7b9011700 1 -- 172.21.15.49:0/1314639045 wait complete. 2024-03-26T18:00:12.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.810+0000 7fc7b9011700 1 Processor -- start 2024-03-26T18:00:12.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.810+0000 7fc7b9011700 1 -- start start 2024-03-26T18:00:12.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.810+0000 7fc7b9011700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4101300 0x7fc7b406b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:12.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.810+0000 7fc7b9011700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc7b4102500 0x7fc7b419d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:12.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b9011700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4102ea0 0x7fc7b419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b9011700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b410cec0 con 0x7fc7b4101300 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b9011700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b410cd40 con 0x7fc7b4102ea0 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b9011700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b410d040 con 0x7fc7b4102500 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4101300 0x7fc7b406b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc7b4102500 0x7fc7b419d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b359e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4102ea0 0x7fc7b419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4101300 0x7fc7b406b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45708/0 (socket says 172.21.15.49:45708) 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b2d9d700 1 -- 172.21.15.49:0/884313295 learned_addr learned my addr 172.21.15.49:0/884313295 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b259c700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc7b4102500 0x7fc7b419d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:44586/0 (socket says 172.21.15.49:44586) 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b259c700 1 -- 172.21.15.49:0/884313295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4102ea0 msgr2=0x7fc7b419f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b259c700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4102ea0 0x7fc7b419f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:12.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b259c700 1 -- 172.21.15.49:0/884313295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4101300 msgr2=0x7fc7b406b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b259c700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4101300 0x7fc7b406b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b259c700 1 -- 172.21.15.49:0/884313295 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b41a01f0 con 0x7fc7b4102500 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b359e700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4102ea0 0x7fc7b419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b2d9d700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4101300 0x7fc7b406b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7b259c700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc7b4102500 0x7fc7b419d2e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00e9f0 tx=0x7fc79c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.811+0000 7fc7abfff700 1 -- 172.21.15.49:0/884313295 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00c9c0 con 0x7fc7b4102500 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.812+0000 7fc7b9011700 1 -- 172.21.15.49:0/884313295 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b41a04e0 con 0x7fc7b4102500 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.812+0000 7fc7abfff700 1 -- 172.21.15.49:0/884313295 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc79c00cb60 con 0x7fc7b4102500 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.812+0000 7fc7abfff700 1 -- 172.21.15.49:0/884313295 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c018680 con 0x7fc7b4102500 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.812+0000 7fc7b9011700 1 -- 172.21.15.49:0/884313295 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b41a0a20 con 0x7fc7b4102500 2024-03-26T18:00:12.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.813+0000 7fc7b9011700 1 -- 172.21.15.49:0/884313295 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b40508c0 con 0x7fc7b4102500 2024-03-26T18:00:12.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.814+0000 7fc7abfff700 1 -- 172.21.15.49:0/884313295 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fc79c010430 con 0x7fc7b4102500 2024-03-26T18:00:12.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.814+0000 7fc7abfff700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7a00a30f0 0x7fc7a00a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:12.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.814+0000 7fc7b2d9d700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7a00a30f0 0x7fc7a00a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:12.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.814+0000 7fc7abfff700 1 -- 172.21.15.49:0/884313295 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc79c0cf8e0 con 0x7fc7b4102500 2024-03-26T18:00:12.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.815+0000 7fc7b2d9d700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7a00a30f0 0x7fc7a00a55b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4000c00 tx=0x7fc7a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:12.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:12.819+0000 7fc7abfff700 1 -- 172.21.15.49:0/884313295 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc79c09c370 con 0x7fc7b4102500 2024-03-26T18:00:12.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:12 smithi171 ceph-mon[36033]: from='client.24827 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:00:12.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:12 smithi171 ceph-mon[36033]: pgmap v304: 129 pgs: 129 active+clean; 621 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.0 KiB/s wr, 13 op/s 2024-03-26T18:00:12.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:12 smithi115 ceph-mon[36174]: from='client.24827 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:00:12.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:12 smithi115 ceph-mon[36174]: pgmap v304: 129 pgs: 129 active+clean; 621 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.0 KiB/s wr, 13 op/s 2024-03-26T18:00:13.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.105+0000 7fc7b9011700 1 -- 172.21.15.49:0/884313295 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc7b4062fc0 con 0x7fc7b4102500 2024-03-26T18:00:13.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.110+0000 7fc7abfff700 1 -- 172.21.15.49:0/884313295 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc79c09bc70 con 0x7fc7b4102500 2024-03-26T18:00:13.094 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-03-26T18:00:13.094 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAtDQNm5dVpBhAAQ3BF0tFBImdWuOEbcqFxfA== 2024-03-26T18:00:13.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 -- 172.21.15.49:0/884313295 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7a00a30f0 msgr2=0x7fc7a00a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:13.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7a00a30f0 0x7fc7a00a55b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4000c00 tx=0x7fc7a4005fb0 comp rx=0 tx=0).stop 2024-03-26T18:00:13.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 -- 172.21.15.49:0/884313295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc7b4102500 msgr2=0x7fc7b419d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:13.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc7b4102500 0x7fc7b419d2e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00e9f0 tx=0x7fc79c00ed00 comp rx=0 tx=0).stop 2024-03-26T18:00:13.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 -- 172.21.15.49:0/884313295 shutdown_connections 2024-03-26T18:00:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7a00a30f0 0x7fc7a00a55b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4102ea0 0x7fc7b419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc7b4101300 0x7fc7b406b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 --2- 172.21.15.49:0/884313295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc7b4102500 0x7fc7b419d2e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 -- 172.21.15.49:0/884313295 >> 172.21.15.49:0/884313295 conn(0x7fc7b40fc8a0 msgr2=0x7fc7b4105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 -- 172.21.15.49:0/884313295 shutdown_connections 2024-03-26T18:00:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:13.112+0000 7fc7b9011700 1 -- 172.21.15.49:0/884313295 wait complete. 2024-03-26T18:00:13.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:13 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/884313295' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:00:13.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:13 smithi049 ceph-mon[32331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:00:13.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:13 smithi049 ceph-mon[32331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T18:00:13.853 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T18:00:13.853 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-26T18:00:13.853 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-26T18:00:13.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:13 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/884313295' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:00:13.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:13 smithi171 ceph-mon[36033]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:00:13.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:13 smithi171 ceph-mon[36033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T18:00:13.903 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-26T18:00:13.903 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-26T18:00:13.903 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-26T18:00:13.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:13 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/884313295' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:00:13.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:13 smithi115 ceph-mon[36174]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T18:00:13.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:13 smithi115 ceph-mon[36174]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T18:00:14.194 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:14.637 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:14 smithi049 ceph-mon[32331]: pgmap v305: 129 pgs: 129 active+clean; 623 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.0 KiB/s wr, 20 op/s 2024-03-26T18:00:14.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:14 smithi171 ceph-mon[36033]: pgmap v305: 129 pgs: 129 active+clean; 623 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.0 KiB/s wr, 20 op/s 2024-03-26T18:00:14.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:14 smithi115 ceph-mon[36174]: pgmap v305: 129 pgs: 129 active+clean; 623 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.0 KiB/s wr, 20 op/s 2024-03-26T18:00:15.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.935+0000 7efe51ae8700 1 -- 172.21.15.49:0/4113190030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c1012b0 msgr2=0x7efe4c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:15.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.935+0000 7efe51ae8700 1 --2- 172.21.15.49:0/4113190030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c1012b0 0x7efe4c101710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efe34009990 tx=0x7efe34009ca0 comp rx=0 tx=0).stop 2024-03-26T18:00:15.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.935+0000 7efe51ae8700 1 -- 172.21.15.49:0/4113190030 shutdown_connections 2024-03-26T18:00:15.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.935+0000 7efe51ae8700 1 --2- 172.21.15.49:0/4113190030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4c101c50 0x7efe4c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:15.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.935+0000 7efe51ae8700 1 --2- 172.21.15.49:0/4113190030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4c1072f0 0x7efe4c1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:15.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.935+0000 7efe51ae8700 1 --2- 172.21.15.49:0/4113190030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c1012b0 0x7efe4c101710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:15.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.935+0000 7efe51ae8700 1 -- 172.21.15.49:0/4113190030 >> 172.21.15.49:0/4113190030 conn(0x7efe4c0781e0 msgr2=0x7efe4c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.935+0000 7efe51ae8700 1 -- 172.21.15.49:0/4113190030 shutdown_connections 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.935+0000 7efe51ae8700 1 -- 172.21.15.49:0/4113190030 wait complete. 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.936+0000 7efe51ae8700 1 Processor -- start 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.936+0000 7efe51ae8700 1 -- start start 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe51ae8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c1012b0 0x7efe4c102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe51ae8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4c101c50 0x7efe4c1034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe51ae8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4c1072f0 0x7efe4c1039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe51ae8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c111550 con 0x7efe4c101c50 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe51ae8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1113d0 con 0x7efe4c1012b0 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe51ae8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1116d0 con 0x7efe4c1072f0 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4c101c50 0x7efe4c1034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:15.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c1012b0 0x7efe4c102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:15.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c1012b0 0x7efe4c102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:35156/0 (socket says 172.21.15.49:35156) 2024-03-26T18:00:15.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4b7fe700 1 -- 172.21.15.49:0/3914230928 learned_addr learned my addr 172.21.15.49:0/3914230928 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:15.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4c101c50 0x7efe4c1034a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45726/0 (socket says 172.21.15.49:45726) 2024-03-26T18:00:15.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4bfff700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4c1072f0 0x7efe4c1039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:15.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4b7fe700 1 -- 172.21.15.49:0/3914230928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4c1072f0 msgr2=0x7efe4c1039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:15.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4b7fe700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4c1072f0 0x7efe4c1039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:15.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4b7fe700 1 -- 172.21.15.49:0/3914230928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4c101c50 msgr2=0x7efe4c1034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:15.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4b7fe700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4c101c50 0x7efe4c1034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:15.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4b7fe700 1 -- 172.21.15.49:0/3914230928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c1a7060 con 0x7efe4c1012b0 2024-03-26T18:00:15.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4affd700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4c101c50 0x7efe4c1034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:15.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.937+0000 7efe4b7fe700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c1012b0 0x7efe4c102f60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00e9a0 tx=0x7efe3c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:15.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.938+0000 7efe48ff9700 1 -- 172.21.15.49:0/3914230928 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00ca10 con 0x7efe4c1012b0 2024-03-26T18:00:15.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.938+0000 7efe48ff9700 1 -- 172.21.15.49:0/3914230928 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3c00cbb0 con 0x7efe4c1012b0 2024-03-26T18:00:15.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.938+0000 7efe51ae8700 1 -- 172.21.15.49:0/3914230928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4c1a7350 con 0x7efe4c1012b0 2024-03-26T18:00:15.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.938+0000 7efe48ff9700 1 -- 172.21.15.49:0/3914230928 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c018680 con 0x7efe4c1012b0 2024-03-26T18:00:15.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.938+0000 7efe51ae8700 1 -- 172.21.15.49:0/3914230928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4c1a78b0 con 0x7efe4c1012b0 2024-03-26T18:00:15.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.938+0000 7efe51ae8700 1 -- 172.21.15.49:0/3914230928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4c069070 con 0x7efe4c1012b0 2024-03-26T18:00:15.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.940+0000 7efe48ff9700 1 -- 172.21.15.49:0/3914230928 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7efe3c010430 con 0x7efe4c1012b0 2024-03-26T18:00:15.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.940+0000 7efe48ff9700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efe380a30c0 0x7efe380a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:15.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.940+0000 7efe4affd700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efe380a30c0 0x7efe380a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:15.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.941+0000 7efe48ff9700 1 -- 172.21.15.49:0/3914230928 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7efe3c0cfc30 con 0x7efe4c1012b0 2024-03-26T18:00:15.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.941+0000 7efe4affd700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efe380a30c0 0x7efe380a5580 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b2e0 tx=0x7efe34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:15.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:15.948+0000 7efe48ff9700 1 -- 172.21.15.49:0/3914230928 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efe3c09c6c0 con 0x7efe4c1012b0 2024-03-26T18:00:16.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.214+0000 7efe51ae8700 1 -- 172.21.15.49:0/3914230928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7efe4c1a7ae0 con 0x7efe4c1012b0 2024-03-26T18:00:16.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.222+0000 7efe48ff9700 1 -- 172.21.15.49:0/3914230928 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263551 (secure 0 0 0) 0x7efe3c09bfc0 con 0x7efe4c1012b0 2024-03-26T18:00:16.209 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:00:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.225+0000 7efe51ae8700 1 -- 172.21.15.49:0/3914230928 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efe380a30c0 msgr2=0x7efe380a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.225+0000 7efe51ae8700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efe380a30c0 0x7efe380a5580 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b2e0 tx=0x7efe34005960 comp rx=0 tx=0).stop 2024-03-26T18:00:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.225+0000 7efe51ae8700 1 -- 172.21.15.49:0/3914230928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c1012b0 msgr2=0x7efe4c102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.225+0000 7efe51ae8700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c1012b0 0x7efe4c102f60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00e9a0 tx=0x7efe3c00ecb0 comp rx=0 tx=0).stop 2024-03-26T18:00:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.225+0000 7efe51ae8700 1 -- 172.21.15.49:0/3914230928 shutdown_connections 2024-03-26T18:00:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.225+0000 7efe51ae8700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efe380a30c0 0x7efe380a5580 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:16.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.225+0000 7efe51ae8700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4c1072f0 0x7efe4c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.225+0000 7efe51ae8700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe4c1012b0 0x7efe4c102f60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.225+0000 7efe51ae8700 1 --2- 172.21.15.49:0/3914230928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe4c101c50 0x7efe4c1034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.225+0000 7efe51ae8700 1 -- 172.21.15.49:0/3914230928 >> 172.21.15.49:0/3914230928 conn(0x7efe4c0781e0 msgr2=0x7efe4c100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.226+0000 7efe51ae8700 1 -- 172.21.15.49:0/3914230928 shutdown_connections 2024-03-26T18:00:16.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:16.226+0000 7efe51ae8700 1 -- 172.21.15.49:0/3914230928 wait complete. 2024-03-26T18:00:16.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:16 smithi171 ceph-mon[36033]: pgmap v306: 129 pgs: 129 active+clean; 623 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2024-03-26T18:00:16.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:16 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3914230928' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T18:00:16.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:16 smithi049 ceph-mon[32331]: pgmap v306: 129 pgs: 129 active+clean; 623 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2024-03-26T18:00:16.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:16 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3914230928' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T18:00:16.920 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":3127469495},{"type":"v1","addr":"172.21.15.49:6801","nonce":3127469495}]},"active_addr":"172.21.15.49:6801/3127469495","active_change":"2024-03-26T17:52:42.621688+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24122,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.49:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1873884295}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1574907427}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1579518314}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":625009684}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3416640100}]}]} 2024-03-26T18:00:16.924 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-26T18:00:16.925 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-26T18:00:16.925 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T18:00:16.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:16 smithi115 ceph-mon[36174]: pgmap v306: 129 pgs: 129 active+clean; 623 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2024-03-26T18:00:16.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:16 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3914230928' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T18:00:17.178 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:18.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:18 smithi115 ceph-mon[36174]: pgmap v307: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-03-26T18:00:18.738 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:18 smithi049 ceph-mon[32331]: pgmap v307: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-03-26T18:00:18.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:18 smithi171 ceph-mon[36033]: pgmap v307: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-03-26T18:00:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.029+0000 7f845e8bd700 1 -- 172.21.15.49:0/2790659149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f845806aa00 msgr2=0x7f845810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.029+0000 7f845e8bd700 1 --2- 172.21.15.49:0/2790659149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f845806aa00 0x7f845810f240 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f845000b0d0 tx=0x7f845000b3e0 comp rx=0 tx=0).stop 2024-03-26T18:00:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.029+0000 7f845e8bd700 1 -- 172.21.15.49:0/2790659149 shutdown_connections 2024-03-26T18:00:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.029+0000 7f845e8bd700 1 --2- 172.21.15.49:0/2790659149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f845806aa00 0x7f845810f240 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.029+0000 7f845e8bd700 1 --2- 172.21.15.49:0/2790659149 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8458108400 0x7f84581087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:19.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.029+0000 7f845e8bd700 1 --2- 172.21.15.49:0/2790659149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f845806a060 0x7f845806a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.029+0000 7f845e8bd700 1 -- 172.21.15.49:0/2790659149 >> 172.21.15.49:0/2790659149 conn(0x7f84580781e0 msgr2=0x7f84580785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.029+0000 7f845e8bd700 1 -- 172.21.15.49:0/2790659149 shutdown_connections 2024-03-26T18:00:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.029+0000 7f845e8bd700 1 -- 172.21.15.49:0/2790659149 wait complete. 2024-03-26T18:00:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.030+0000 7f845e8bd700 1 Processor -- start 2024-03-26T18:00:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.030+0000 7f845e8bd700 1 -- start start 2024-03-26T18:00:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.030+0000 7f845e8bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f845806a060 0x7f845819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.030+0000 7f845e8bd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f845806aa00 0x7f845819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.030+0000 7f845e8bd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8458108400 0x7f8458198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.030+0000 7f845e8bd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8458112680 con 0x7f845806aa00 2024-03-26T18:00:19.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.030+0000 7f845e8bd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8458112500 con 0x7f845806a060 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.030+0000 7f845e8bd700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8458112800 con 0x7f8458108400 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f845806aa00 0x7f845819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f8457fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f845806a060 0x7f845819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f8457fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f845806a060 0x7f845819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:58734/0 (socket says 172.21.15.49:58734) 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f845806aa00 0x7f845819ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53794/0 (socket says 172.21.15.49:53794) 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f8457fff700 1 -- 172.21.15.49:0/2372977376 learned_addr learned my addr 172.21.15.49:0/2372977376 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f845ce5a700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8458108400 0x7f8458198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84577fe700 1 -- 172.21.15.49:0/2372977376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8458108400 msgr2=0x7f8458198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84577fe700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8458108400 0x7f8458198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84577fe700 1 -- 172.21.15.49:0/2372977376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f845806a060 msgr2=0x7f845819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:19.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84577fe700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f845806a060 0x7f845819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84577fe700 1 -- 172.21.15.49:0/2372977376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8458199060 con 0x7f845806aa00 2024-03-26T18:00:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f8457fff700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f845806a060 0x7f845819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84577fe700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f845806aa00 0x7f845819ec80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f844c00e9f0 tx=0x7f844c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84557fa700 1 -- 172.21.15.49:0/2372977376 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c00c9c0 con 0x7f845806aa00 2024-03-26T18:00:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f845e8bd700 1 -- 172.21.15.49:0/2372977376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8458199350 con 0x7f845806aa00 2024-03-26T18:00:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84557fa700 1 -- 172.21.15.49:0/2372977376 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f844c00cb60 con 0x7f845806aa00 2024-03-26T18:00:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f84557fa700 1 -- 172.21.15.49:0/2372977376 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c018680 con 0x7f845806aa00 2024-03-26T18:00:19.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.031+0000 7f845e8bd700 1 -- 172.21.15.49:0/2372977376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84581a5610 con 0x7f845806aa00 2024-03-26T18:00:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.032+0000 7f845e8bd700 1 -- 172.21.15.49:0/2372977376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8458069070 con 0x7f845806aa00 2024-03-26T18:00:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.033+0000 7f84557fa700 1 -- 172.21.15.49:0/2372977376 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f844c010430 con 0x7f845806aa00 2024-03-26T18:00:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.034+0000 7f84557fa700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84400a3040 0x7f84400a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:19.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.034+0000 7f8457fff700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84400a3040 0x7f84400a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:19.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.034+0000 7f84557fa700 1 -- 172.21.15.49:0/2372977376 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f844c0cfa70 con 0x7f845806aa00 2024-03-26T18:00:19.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.035+0000 7f8457fff700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84400a3040 0x7f84400a5500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8458199a40 tx=0x7f8448009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:19.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.038+0000 7f84557fa700 1 -- 172.21.15.49:0/2372977376 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f844c09c500 con 0x7f845806aa00 2024-03-26T18:00:19.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.248+0000 7f845e8bd700 1 -- 172.21.15.49:0/2372977376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f845802dc00 con 0x7f845806aa00 2024-03-26T18:00:19.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.250+0000 7f84557fa700 1 -- 172.21.15.49:0/2372977376 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24136 (secure 0 0 0) 0x7f844c09be00 con 0x7f845806aa00 2024-03-26T18:00:19.232 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:00:19.232 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:51.836345+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:00:19.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 -- 172.21.15.49:0/2372977376 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84400a3040 msgr2=0x7f84400a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:19.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84400a3040 0x7f84400a5500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8458199a40 tx=0x7f8448009260 comp rx=0 tx=0).stop 2024-03-26T18:00:19.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 -- 172.21.15.49:0/2372977376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f845806aa00 msgr2=0x7f845819ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:19.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f845806aa00 0x7f845819ec80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f844c00e9f0 tx=0x7f844c00ed00 comp rx=0 tx=0).stop 2024-03-26T18:00:19.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 -- 172.21.15.49:0/2372977376 shutdown_connections 2024-03-26T18:00:19.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f84400a3040 0x7f84400a5500 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:19.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8458108400 0x7f8458198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:19.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f845806a060 0x7f845819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:19.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 --2- 172.21.15.49:0/2372977376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f845806aa00 0x7f845819ec80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:19.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 -- 172.21.15.49:0/2372977376 >> 172.21.15.49:0/2372977376 conn(0x7f84580781e0 msgr2=0x7f84580ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:19.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 -- 172.21.15.49:0/2372977376 shutdown_connections 2024-03-26T18:00:19.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:19.252+0000 7f845e8bd700 1 -- 172.21.15.49:0/2372977376 wait complete. 2024-03-26T18:00:19.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:19 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2372977376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:19.906 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:19 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2372977376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:19.938 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-26T18:00:19.938 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T18:00:19.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:19 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2372977376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:20.193 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:20.668 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:20 smithi049 ceph-mon[32331]: pgmap v308: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2024-03-26T18:00:20.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:20 smithi171 ceph-mon[36033]: pgmap v308: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2024-03-26T18:00:20.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:20 smithi115 ceph-mon[36174]: pgmap v308: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2024-03-26T18:00:21.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.987+0000 7f85b5cfc700 1 -- 172.21.15.49:0/3511964025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b01065d0 msgr2=0x7f85b010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.987+0000 7f85b5cfc700 1 --2- 172.21.15.49:0/3511964025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b01065d0 0x7f85b010e1a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f85a800b0a0 tx=0x7f85a800b3b0 comp rx=0 tx=0).stop 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.987+0000 7f85b5cfc700 1 -- 172.21.15.49:0/3511964025 shutdown_connections 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.987+0000 7f85b5cfc700 1 --2- 172.21.15.49:0/3511964025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b01065d0 0x7f85b010e1a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.987+0000 7f85b5cfc700 1 --2- 172.21.15.49:0/3511964025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b0101540 0x7f85b0101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.987+0000 7f85b5cfc700 1 --2- 172.21.15.49:0/3511964025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85b0101ef0 0x7f85b0105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.987+0000 7f85b5cfc700 1 -- 172.21.15.49:0/3511964025 >> 172.21.15.49:0/3511964025 conn(0x7f85b00772d0 msgr2=0x7f85b00776e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.987+0000 7f85b5cfc700 1 -- 172.21.15.49:0/3511964025 shutdown_connections 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.987+0000 7f85b5cfc700 1 -- 172.21.15.49:0/3511964025 wait complete. 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.987+0000 7f85b5cfc700 1 Processor -- start 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85b5cfc700 1 -- start start 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85b5cfc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0101540 0x7f85b019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:21.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85b5cfc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85b0101ef0 0x7f85b019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85b5cfc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b01065d0 0x7f85b01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85b5cfc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01115b0 con 0x7f85b0101540 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85b5cfc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0111430 con 0x7f85b01065d0 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85b5cfc700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0111730 con 0x7f85b0101ef0 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85af7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0101540 0x7f85b019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85af7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0101540 0x7f85b019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53822/0 (socket says 172.21.15.49:53822) 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85af7fe700 1 -- 172.21.15.49:0/2964976474 learned_addr learned my addr 172.21.15.49:0/2964976474 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85aeffd700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85b0101ef0 0x7f85b019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85affff700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b01065d0 0x7f85b01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85af7fe700 1 -- 172.21.15.49:0/2964976474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85b0101ef0 msgr2=0x7f85b019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85af7fe700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85b0101ef0 0x7f85b019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85af7fe700 1 -- 172.21.15.49:0/2964976474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b01065d0 msgr2=0x7f85b01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:21.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85af7fe700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b01065d0 0x7f85b01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:21.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.988+0000 7f85af7fe700 1 -- 172.21.15.49:0/2964976474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b0199100 con 0x7f85b0101540 2024-03-26T18:00:21.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.989+0000 7f85af7fe700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0101540 0x7f85b019e7e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85a000eb30 tx=0x7f85a000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:21.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.989+0000 7f85acff9700 1 -- 172.21.15.49:0/2964976474 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a000cbd0 con 0x7f85b0101540 2024-03-26T18:00:21.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.989+0000 7f85acff9700 1 -- 172.21.15.49:0/2964976474 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a0004500 con 0x7f85b0101540 2024-03-26T18:00:21.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.989+0000 7f85b5cfc700 1 -- 172.21.15.49:0/2964976474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b01993f0 con 0x7f85b0101540 2024-03-26T18:00:21.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.989+0000 7f85acff9700 1 -- 172.21.15.49:0/2964976474 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0010430 con 0x7f85b0101540 2024-03-26T18:00:21.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.989+0000 7f85b5cfc700 1 -- 172.21.15.49:0/2964976474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b01a56b0 con 0x7f85b0101540 2024-03-26T18:00:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.990+0000 7f85b5cfc700 1 -- 172.21.15.49:0/2964976474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b01029d0 con 0x7f85b0101540 2024-03-26T18:00:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.991+0000 7f85acff9700 1 -- 172.21.15.49:0/2964976474 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f85a0003f00 con 0x7f85b0101540 2024-03-26T18:00:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.991+0000 7f85acff9700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f85980a3040 0x7f85980a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.992+0000 7f85acff9700 1 -- 172.21.15.49:0/2964976474 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f85a00cfb20 con 0x7f85b0101540 2024-03-26T18:00:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.992+0000 7f85aeffd700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f85980a3040 0x7f85980a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:21.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.992+0000 7f85aeffd700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f85980a3040 0x7f85980a5500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f85b0107b00 tx=0x7f85a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:21.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:21.996+0000 7f85acff9700 1 -- 172.21.15.49:0/2964976474 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f85a009c5b0 con 0x7f85b0101540 2024-03-26T18:00:22.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.201+0000 7f85b5cfc700 1 -- 172.21.15.49:0/2964976474 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85b0199ed0 con 0x7f85b0101540 2024-03-26T18:00:22.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.202+0000 7f85acff9700 1 -- 172.21.15.49:0/2964976474 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24136 (secure 0 0 0) 0x7f85a009beb0 con 0x7f85b0101540 2024-03-26T18:00:22.185 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:00:22.185 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T17:59:51.836345+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.204+0000 7f85b5cfc700 1 -- 172.21.15.49:0/2964976474 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f85980a3040 msgr2=0x7f85980a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.204+0000 7f85b5cfc700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f85980a3040 0x7f85980a5500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f85b0107b00 tx=0x7f85a4009260 comp rx=0 tx=0).stop 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.204+0000 7f85b5cfc700 1 -- 172.21.15.49:0/2964976474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0101540 msgr2=0x7f85b019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.204+0000 7f85b5cfc700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0101540 0x7f85b019e7e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85a000eb30 tx=0x7f85a000ee40 comp rx=0 tx=0).stop 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.204+0000 7f85b5cfc700 1 -- 172.21.15.49:0/2964976474 shutdown_connections 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.204+0000 7f85b5cfc700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f85980a3040 0x7f85980a5500 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.204+0000 7f85b5cfc700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b01065d0 0x7f85b01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.204+0000 7f85b5cfc700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85b0101540 0x7f85b019e7e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.204+0000 7f85b5cfc700 1 --2- 172.21.15.49:0/2964976474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85b0101ef0 0x7f85b019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.204+0000 7f85b5cfc700 1 -- 172.21.15.49:0/2964976474 >> 172.21.15.49:0/2964976474 conn(0x7f85b00772d0 msgr2=0x7f85b006ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.205+0000 7f85b5cfc700 1 -- 172.21.15.49:0/2964976474 shutdown_connections 2024-03-26T18:00:22.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:22.205+0000 7f85b5cfc700 1 -- 172.21.15.49:0/2964976474 wait complete. 2024-03-26T18:00:22.594 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:22 smithi049 ceph-mon[32331]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-03-26T18:00:22.594 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:22 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2964976474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:22.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:22 smithi171 ceph-mon[36033]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-03-26T18:00:22.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:22 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2964976474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:22.986 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-26T18:00:22.987 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-26T18:00:22.987 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-26T18:00:22.988 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-26T18:00:22.988 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-26T18:00:22.989 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-26T18:00:22.989 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-26T18:00:22.989 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-26T18:00:22.990 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-26T18:00:22.990 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-26T18:00:22.991 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-26T18:00:22.991 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-26T18:00:22.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:22 smithi115 ceph-mon[36174]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-03-26T18:00:22.994 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:22 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2964976474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:00:23.723 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:23.727 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:23.729 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:23.737 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:23.743 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:24.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:24 smithi049 ceph-mon[32331]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-03-26T18:00:24.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:24 smithi171 ceph-mon[36033]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-03-26T18:00:24.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:24 smithi115 ceph-mon[36174]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-03-26T18:00:26.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:26 smithi171 ceph-mon[36033]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-03-26T18:00:26.964 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:26 smithi049 ceph-mon[32331]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-03-26T18:00:26.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:26 smithi115 ceph-mon[36174]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-03-26T18:00:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:27.363 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:27.468 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:28.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:28 smithi171 ceph-mon[36033]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-03-26T18:00:28.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:28 smithi115 ceph-mon[36174]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-03-26T18:00:28.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:28 smithi049 ceph-mon[32331]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-03-26T18:00:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.261+0000 7fb561f2a700 1 -- 172.21.15.49:0/1029958809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb55c1036c0 msgr2=0x7fb55c103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.261+0000 7fb561f2a700 1 --2- 172.21.15.49:0/1029958809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb55c1036c0 0x7fb55c103aa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb544009990 tx=0x7fb544009ca0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.261+0000 7fb561f2a700 1 -- 172.21.15.49:0/1029958809 shutdown_connections 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.261+0000 7fb561f2a700 1 --2- 172.21.15.49:0/1029958809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb55c10caa0 0x7fb55c10eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.261+0000 7fb561f2a700 1 --2- 172.21.15.49:0/1029958809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb55c1036c0 0x7fb55c103aa0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.261+0000 7fb561f2a700 1 --2- 172.21.15.49:0/1029958809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb55c103fe0 0x7fb55c10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.261+0000 7fb561f2a700 1 -- 172.21.15.49:0/1029958809 >> 172.21.15.49:0/1029958809 conn(0x7fb55c0783e0 msgr2=0x7fb55c0787f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.261+0000 7fb561f2a700 1 -- 172.21.15.49:0/1029958809 shutdown_connections 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.261+0000 7fb561f2a700 1 -- 172.21.15.49:0/1029958809 wait complete. 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.262+0000 7fb561f2a700 1 Processor -- start 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.262+0000 7fb561f2a700 1 -- start start 2024-03-26T18:00:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.262+0000 7fb561f2a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb55c1036c0 0x7fb55c101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.262+0000 7fb561f2a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb55c103fe0 0x7fb55c101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.262+0000 7fb561f2a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb55c10caa0 0x7fb55c101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.262+0000 7fb561f2a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c1127a0 con 0x7fb55c103fe0 2024-03-26T18:00:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.262+0000 7fb561f2a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c112620 con 0x7fb55c10caa0 2024-03-26T18:00:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.262+0000 7fb561f2a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c112920 con 0x7fb55c1036c0 2024-03-26T18:00:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.262+0000 7fb55affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb55c103fe0 0x7fb55c101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55b7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb55c1036c0 0x7fb55c101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb55c103fe0 0x7fb55c101860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58038/0 (socket says 172.21.15.49:58038) 2024-03-26T18:00:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb55c10caa0 0x7fb55c101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55affd700 1 -- 172.21.15.49:0/404641241 learned_addr learned my addr 172.21.15.49:0/404641241 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55affd700 1 -- 172.21.15.49:0/404641241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb55c1036c0 msgr2=0x7fb55c101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55affd700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb55c1036c0 0x7fb55c101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55affd700 1 -- 172.21.15.49:0/404641241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb55c10caa0 msgr2=0x7fb55c101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55affd700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb55c10caa0 0x7fb55c101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55affd700 1 -- 172.21.15.49:0/404641241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55c1a7160 con 0x7fb55c103fe0 2024-03-26T18:00:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55bfff700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb55c10caa0 0x7fb55c101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55b7fe700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb55c1036c0 0x7fb55c101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb55affd700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb55c103fe0 0x7fb55c101860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00e9f0 tx=0x7fb54c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb558ff9700 1 -- 172.21.15.49:0/404641241 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c004a30 con 0x7fb55c103fe0 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb558ff9700 1 -- 172.21.15.49:0/404641241 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb54c00cca0 con 0x7fb55c103fe0 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55c1a7450 con 0x7fb55c103fe0 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb558ff9700 1 -- 172.21.15.49:0/404641241 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0056c0 con 0x7fb55c103fe0 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.263+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55c1a7940 con 0x7fb55c103fe0 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.264+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb55c062ac0 con 0x7fb55c103fe0 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.266+0000 7fb558ff9700 1 -- 172.21.15.49:0/404641241 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fb54c00c7c0 con 0x7fb55c103fe0 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.266+0000 7fb558ff9700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb5480a3040 0x7fb5480a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.266+0000 7fb55b7fe700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb5480a3040 0x7fb5480a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.266+0000 7fb558ff9700 1 -- 172.21.15.49:0/404641241 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb54c0cfdd0 con 0x7fb55c103fe0 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.266+0000 7fb558ff9700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] conn(0x7fb5480add00 0x7fb5480b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.266+0000 7fb558ff9700 1 -- 172.21.15.49:0/404641241 --> [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb54c004500 con 0x7fb5480add00 2024-03-26T18:00:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.266+0000 7fb558ff9700 1 -- 172.21.15.49:0/404641241 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb54c0d0220 con 0x7fb55c103fe0 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.267+0000 7fb55bfff700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] conn(0x7fb5480add00 0x7fb5480b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.267+0000 7fb55b7fe700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb5480a3040 0x7fb5480a5500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb54400b2e0 tx=0x7fb544005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.267+0000 7fb55bfff700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] conn(0x7fb5480add00 0x7fb5480b0140 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.268+0000 7fb558ff9700 1 -- 172.21.15.49:0/404641241 <== osd.3 v2:172.21.15.49:6810/3710621844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb54c004500 con 0x7fb5480add00 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.284+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 --> [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb55c062ec0 con 0x7fb5480add00 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.284+0000 7fb558ff9700 1 -- 172.21.15.49:0/404641241 <== osd.3 v2:172.21.15.49:6810/3710621844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb55c062ec0 con 0x7fb5480add00 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.284+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] conn(0x7fb5480add00 msgr2=0x7fb5480b0140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.285+0000 7fb561f2a700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] conn(0x7fb5480add00 0x7fb5480b0140 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.285+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb5480a3040 msgr2=0x7fb5480a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.285+0000 7fb561f2a700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb5480a3040 0x7fb5480a5500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb54400b2e0 tx=0x7fb544005960 comp rx=0 tx=0).stop 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.285+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb55c103fe0 msgr2=0x7fb55c101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.285+0000 7fb561f2a700 1 --2- 172.21.15.49:0/404641241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb55c103fe0 0x7fb55c101860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00e9f0 tx=0x7fb54c00ed00 comp rx=0 tx=0).stop 2024-03-26T18:00:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.285+0000 7fb55bfff700 1 -- 172.21.15.49:0/404641241 reap_dead start 2024-03-26T18:00:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.285+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 shutdown_connections 2024-03-26T18:00:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.285+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 >> 172.21.15.49:0/404641241 conn(0x7fb55c0783e0 msgr2=0x7fb55c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.285+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 shutdown_connections 2024-03-26T18:00:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:30.285+0000 7fb561f2a700 1 -- 172.21.15.49:0/404641241 wait complete. 2024-03-26T18:00:30.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:30 smithi171 ceph-mon[36033]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-26T18:00:30.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:30 smithi115 ceph-mon[36174]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-26T18:00:31.002 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:30 smithi049 ceph-mon[32331]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-26T18:00:32.204 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:32.205 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:32.205 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:32.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:32 smithi049 ceph-mon[32331]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-26T18:00:32.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:32 smithi171 ceph-mon[36033]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-26T18:00:32.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:32 smithi115 ceph-mon[36174]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-26T18:00:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.132+0000 7fc0b41a6700 1 -- 172.21.15.49:0/13207150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ac1072a0 msgr2=0x7fc0ac06a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.132+0000 7fc0b41a6700 1 --2- 172.21.15.49:0/13207150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ac1072a0 0x7fc0ac06a060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009990 tx=0x7fc0a8009ca0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.133+0000 7fc0b41a6700 1 -- 172.21.15.49:0/13207150 shutdown_connections 2024-03-26T18:00:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.133+0000 7fc0b41a6700 1 --2- 172.21.15.49:0/13207150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac104db0 0x7fc0ac105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.133+0000 7fc0b41a6700 1 --2- 172.21.15.49:0/13207150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ac1072a0 0x7fc0ac06a060 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.133+0000 7fc0b41a6700 1 --2- 172.21.15.49:0/13207150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0ac06a5a0 0x7fc0ac06aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.133+0000 7fc0b41a6700 1 -- 172.21.15.49:0/13207150 >> 172.21.15.49:0/13207150 conn(0x7fc0ac0781e0 msgr2=0x7fc0ac0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.133+0000 7fc0b41a6700 1 -- 172.21.15.49:0/13207150 shutdown_connections 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.133+0000 7fc0b41a6700 1 -- 172.21.15.49:0/13207150 wait complete. 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b41a6700 1 Processor -- start 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b41a6700 1 -- start start 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b41a6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0ac06a5a0 0x7fc0ac19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b41a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac104db0 0x7fc0ac19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b41a6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ac1072a0 0x7fc0ac1a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b41a6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac118c50 con 0x7fc0ac1072a0 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b41a6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac118ad0 con 0x7fc0ac104db0 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b41a6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac118dd0 con 0x7fc0ac06a5a0 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b1f42700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0ac06a5a0 0x7fc0ac19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b1741700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac104db0 0x7fc0ac19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b1741700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac104db0 0x7fc0ac19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:47716/0 (socket says 172.21.15.49:47716) 2024-03-26T18:00:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b1f42700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0ac06a5a0 0x7fc0ac19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:37506/0 (socket says 172.21.15.49:37506) 2024-03-26T18:00:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b1741700 1 -- 172.21.15.49:0/2834286484 learned_addr learned my addr 172.21.15.49:0/2834286484 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.134+0000 7fc0b2743700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ac1072a0 0x7fc0ac1a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0b1741700 1 -- 172.21.15.49:0/2834286484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0ac06a5a0 msgr2=0x7fc0ac19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0b1741700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0ac06a5a0 0x7fc0ac19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0b1741700 1 -- 172.21.15.49:0/2834286484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ac1072a0 msgr2=0x7fc0ac1a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0b1741700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0ac1072a0 0x7fc0ac1a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0b1741700 1 -- 172.21.15.49:0/2834286484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac1a4110 con 0x7fc0ac104db0 2024-03-26T18:00:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0b1f42700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0ac06a5a0 0x7fc0ac19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:33.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0b1741700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac104db0 0x7fc0ac19f640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400ef30 tx=0x7fc0a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0a2ffd700 1 -- 172.21.15.49:0/2834286484 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40092e0 con 0x7fc0ac104db0 2024-03-26T18:00:33.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0a2ffd700 1 -- 172.21.15.49:0/2834286484 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a4004bb0 con 0x7fc0ac104db0 2024-03-26T18:00:33.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac1a43a0 con 0x7fc0ac104db0 2024-03-26T18:00:33.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0a2ffd700 1 -- 172.21.15.49:0/2834286484 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4004280 con 0x7fc0ac104db0 2024-03-26T18:00:33.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.135+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac1abc80 con 0x7fc0ac104db0 2024-03-26T18:00:33.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.136+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc0ac100d00 con 0x7fc0ac104db0 2024-03-26T18:00:33.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.137+0000 7fc0a2ffd700 1 -- 172.21.15.49:0/2834286484 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fc0a401b030 con 0x7fc0ac104db0 2024-03-26T18:00:33.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.138+0000 7fc0a2ffd700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc0980a30c0 0x7fc0980a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.138+0000 7fc0b1f42700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc0980a30c0 0x7fc0980a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.138+0000 7fc0a2ffd700 1 -- 172.21.15.49:0/2834286484 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc0a40cfc10 con 0x7fc0ac104db0 2024-03-26T18:00:33.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.138+0000 7fc0a2ffd700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] conn(0x7fc0980add80 0x7fc0980b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.138+0000 7fc0a2ffd700 1 -- 172.21.15.49:0/2834286484 --> [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0980b0890 con 0x7fc0980add80 2024-03-26T18:00:33.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.138+0000 7fc0a2ffd700 1 -- 172.21.15.49:0/2834286484 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc0a40d0060 con 0x7fc0ac104db0 2024-03-26T18:00:33.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.139+0000 7fc0b2743700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] conn(0x7fc0980add80 0x7fc0980b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.139+0000 7fc0b1f42700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc0980a30c0 0x7fc0980a5580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800b2e0 tx=0x7fc0a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.139+0000 7fc0b2743700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] conn(0x7fc0980add80 0x7fc0980b01c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.140+0000 7fc0a2ffd700 1 -- 172.21.15.49:0/2834286484 <== osd.11 v2:172.21.15.171:6824/4157105640 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc0980b0890 con 0x7fc0980add80 2024-03-26T18:00:33.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.156+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 --> [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc0ac101890 con 0x7fc0980add80 2024-03-26T18:00:33.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0a2ffd700 1 -- 172.21.15.49:0/2834286484 <== osd.11 v2:172.21.15.171:6824/4157105640 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc0ac101890 con 0x7fc0980add80 2024-03-26T18:00:33.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 >> [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] conn(0x7fc0980add80 msgr2=0x7fc0980b01c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b41a6700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] conn(0x7fc0980add80 0x7fc0980b01c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc0980a30c0 msgr2=0x7fc0980a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b41a6700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc0980a30c0 0x7fc0980a5580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800b2e0 tx=0x7fc0a8005960 comp rx=0 tx=0).stop 2024-03-26T18:00:33.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac104db0 msgr2=0x7fc0ac19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b41a6700 1 --2- 172.21.15.49:0/2834286484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac104db0 0x7fc0ac19f640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400ef30 tx=0x7fc0a400c3a0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b2743700 1 -- 172.21.15.49:0/2834286484 reap_dead start 2024-03-26T18:00:33.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 shutdown_connections 2024-03-26T18:00:33.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 >> 172.21.15.49:0/2834286484 conn(0x7fc0ac0781e0 msgr2=0x7fc0ac1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:33.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 shutdown_connections 2024-03-26T18:00:33.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.157+0000 7fc0b41a6700 1 -- 172.21.15.49:0/2834286484 wait complete. 2024-03-26T18:00:33.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.851+0000 7fd77e508700 1 -- 172.21.15.49:0/1216372704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7781076d0 msgr2=0x7fd77810f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.851+0000 7fd77e508700 1 --2- 172.21.15.49:0/1216372704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7781076d0 0x7fd77810f2a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd77000b0d0 tx=0x7fd77000b3e0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.851+0000 7fd77e508700 1 -- 172.21.15.49:0/1216372704 shutdown_connections 2024-03-26T18:00:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.851+0000 7fd77e508700 1 --2- 172.21.15.49:0/1216372704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd7781076d0 0x7fd77810f2a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.851+0000 7fd77e508700 1 --2- 172.21.15.49:0/1216372704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7781025f0 0x7fd7781029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.851+0000 7fd77e508700 1 --2- 172.21.15.49:0/1216372704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd778102fa0 0x7fd778106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.851+0000 7fd77e508700 1 -- 172.21.15.49:0/1216372704 >> 172.21.15.49:0/1216372704 conn(0x7fd7780fde60 msgr2=0x7fd778100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.851+0000 7fd77e508700 1 -- 172.21.15.49:0/1216372704 shutdown_connections 2024-03-26T18:00:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.851+0000 7fd77e508700 1 -- 172.21.15.49:0/1216372704 wait complete. 2024-03-26T18:00:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.852+0000 7fd77e508700 1 Processor -- start 2024-03-26T18:00:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.852+0000 7fd77e508700 1 -- start start 2024-03-26T18:00:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.852+0000 7fd77e508700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7781025f0 0x7fd77819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.852+0000 7fd77e508700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd778102fa0 0x7fd77819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.852+0000 7fd77e508700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7781076d0 0x7fd778198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.852+0000 7fd77e508700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd778112710 con 0x7fd778102fa0 2024-03-26T18:00:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.852+0000 7fd77e508700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd778112590 con 0x7fd7781025f0 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.852+0000 7fd77e508700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd778112890 con 0x7fd7781076d0 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd7777fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd778102fa0 0x7fd77819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd777fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7781025f0 0x7fd77819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd7777fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd778102fa0 0x7fd77819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58072/0 (socket says 172.21.15.49:58072) 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd777fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7781025f0 0x7fd77819e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:47732/0 (socket says 172.21.15.49:47732) 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd7777fe700 1 -- 172.21.15.49:0/246350483 learned_addr learned my addr 172.21.15.49:0/246350483 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd77caa5700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7781076d0 0x7fd778198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd777fff700 1 -- 172.21.15.49:0/246350483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7781076d0 msgr2=0x7fd778198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd777fff700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7781076d0 0x7fd778198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd777fff700 1 -- 172.21.15.49:0/246350483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd778102fa0 msgr2=0x7fd77819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd777fff700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd778102fa0 0x7fd77819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd777fff700 1 -- 172.21.15.49:0/246350483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd778199090 con 0x7fd7781025f0 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd777fff700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7781025f0 0x7fd77819e770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd76800e9f0 tx=0x7fd76800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd7757fa700 1 -- 172.21.15.49:0/246350483 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76800c9c0 con 0x7fd7781025f0 2024-03-26T18:00:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd7757fa700 1 -- 172.21.15.49:0/246350483 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd76800cb60 con 0x7fd7781025f0 2024-03-26T18:00:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd778199380 con 0x7fd7781025f0 2024-03-26T18:00:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd7757fa700 1 -- 172.21.15.49:0/246350483 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768018680 con 0x7fd7781025f0 2024-03-26T18:00:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.853+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7781a5640 con 0x7fd7781025f0 2024-03-26T18:00:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.854+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd778062b40 con 0x7fd7781025f0 2024-03-26T18:00:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.855+0000 7fd7757fa700 1 -- 172.21.15.49:0/246350483 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fd768010430 con 0x7fd7781025f0 2024-03-26T18:00:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.856+0000 7fd7757fa700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7600a3040 0x7fd7600a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.856+0000 7fd7777fe700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7600a3040 0x7fd7600a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.856+0000 7fd7757fa700 1 -- 172.21.15.49:0/246350483 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd7680cfc70 con 0x7fd7781025f0 2024-03-26T18:00:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.856+0000 7fd7757fa700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] conn(0x7fd7600add00 0x7fd7600b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.856+0000 7fd7757fa700 1 -- 172.21.15.49:0/246350483 --> [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7600b0810 con 0x7fd7600add00 2024-03-26T18:00:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.856+0000 7fd7757fa700 1 -- 172.21.15.49:0/246350483 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd7680d0020 con 0x7fd7781025f0 2024-03-26T18:00:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.856+0000 7fd77caa5700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] conn(0x7fd7600add00 0x7fd7600b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.856+0000 7fd7777fe700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7600a3040 0x7fd7600a5500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd778108c50 tx=0x7fd76c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.857+0000 7fd77caa5700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] conn(0x7fd7600add00 0x7fd7600b0140 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.857+0000 7fd7757fa700 1 -- 172.21.15.49:0/246350483 <== osd.4 v2:172.21.15.115:6808/1015737283 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7600b0810 con 0x7fd7600add00 2024-03-26T18:00:33.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.874+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 --> [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd778062fc0 con 0x7fd7600add00 2024-03-26T18:00:33.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.874+0000 7fd7757fa700 1 -- 172.21.15.49:0/246350483 <== osd.4 v2:172.21.15.115:6808/1015737283 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd778062fc0 con 0x7fd7600add00 2024-03-26T18:00:33.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.874+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 >> [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] conn(0x7fd7600add00 msgr2=0x7fd7600b0140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.874+0000 7fd77e508700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] conn(0x7fd7600add00 0x7fd7600b0140 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.874+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7600a3040 msgr2=0x7fd7600a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.874+0000 7fd77e508700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd7600a3040 0x7fd7600a5500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd778108c50 tx=0x7fd76c009260 comp rx=0 tx=0).stop 2024-03-26T18:00:33.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.874+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7781025f0 msgr2=0x7fd77819e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.874+0000 7fd77e508700 1 --2- 172.21.15.49:0/246350483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7781025f0 0x7fd77819e770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd76800e9f0 tx=0x7fd76800ed00 comp rx=0 tx=0).stop 2024-03-26T18:00:33.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.874+0000 7fd77caa5700 1 -- 172.21.15.49:0/246350483 reap_dead start 2024-03-26T18:00:33.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.875+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 shutdown_connections 2024-03-26T18:00:33.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.875+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 >> 172.21.15.49:0/246350483 conn(0x7fd7780fde60 msgr2=0x7fd7780fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:33.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.875+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 shutdown_connections 2024-03-26T18:00:33.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:33.875+0000 7fd77e508700 1 -- 172.21.15.49:0/246350483 wait complete. 2024-03-26T18:00:34.826 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:34 smithi049 ceph-mon[32331]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-26T18:00:34.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:34 smithi171 ceph-mon[36033]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-26T18:00:34.860 INFO:teuthology.orchestra.run.smithi049.stdout:146028888122 2024-03-26T18:00:34.861 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-26T18:00:34.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:34 smithi115 ceph-mon[36174]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-26T18:00:36.718 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:36 smithi049 ceph-mon[32331]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-26T18:00:36.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:36 smithi171 ceph-mon[36033]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-26T18:00:36.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:36 smithi115 ceph-mon[36174]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-26T18:00:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.521+0000 7f8ba67d3700 1 -- 172.21.15.49:0/1146838153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ba01026c0 msgr2=0x7f8ba0105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.521+0000 7f8ba67d3700 1 --2- 172.21.15.49:0/1146838153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ba01026c0 0x7f8ba0105f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b940099c0 tx=0x7f8b94009cd0 comp rx=0 tx=0).stop 2024-03-26T18:00:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.521+0000 7f8ba67d3700 1 -- 172.21.15.49:0/1146838153 shutdown_connections 2024-03-26T18:00:37.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.521+0000 7f8ba67d3700 1 --2- 172.21.15.49:0/1146838153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba0106630 0x7f8ba010e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.521+0000 7f8ba67d3700 1 --2- 172.21.15.49:0/1146838153 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba0101d10 0x7f8ba01020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.521+0000 7f8ba67d3700 1 --2- 172.21.15.49:0/1146838153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ba01026c0 0x7f8ba0105f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.521+0000 7f8ba67d3700 1 -- 172.21.15.49:0/1146838153 >> 172.21.15.49:0/1146838153 conn(0x7f8ba00778b0 msgr2=0x7f8ba00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.522+0000 7f8ba67d3700 1 -- 172.21.15.49:0/1146838153 shutdown_connections 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.522+0000 7f8ba67d3700 1 -- 172.21.15.49:0/1146838153 wait complete. 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.522+0000 7f8ba67d3700 1 Processor -- start 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.522+0000 7f8ba67d3700 1 -- start start 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba67d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba0101d10 0x7f8ba0199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba67d3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba01026c0 0x7f8ba019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba67d3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ba0106630 0x7f8ba01a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba67d3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0111670 con 0x7f8ba01026c0 2024-03-26T18:00:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba67d3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01114f0 con 0x7f8ba0101d10 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba67d3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01117f0 con 0x7f8ba0106630 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba4d70700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ba0106630 0x7f8ba01a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8b9ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba0101d10 0x7f8ba0199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8b9f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba01026c0 0x7f8ba019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba4d70700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ba0106630 0x7f8ba01a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:37552/0 (socket says 172.21.15.49:37552) 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8b9ffff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba0101d10 0x7f8ba0199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:47764/0 (socket says 172.21.15.49:47764) 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba4d70700 1 -- 172.21.15.49:0/2749321588 learned_addr learned my addr 172.21.15.49:0/2749321588 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba4d70700 1 -- 172.21.15.49:0/2749321588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba0101d10 msgr2=0x7f8ba0199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba4d70700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba0101d10 0x7f8ba0199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba4d70700 1 -- 172.21.15.49:0/2749321588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba01026c0 msgr2=0x7f8ba019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:37.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba4d70700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba01026c0 0x7f8ba019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8ba4d70700 1 -- 172.21.15.49:0/2749321588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba01a50d0 con 0x7f8ba0106630 2024-03-26T18:00:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.523+0000 7f8b9f7fe700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba01026c0 0x7f8ba019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.524+0000 7f8ba4d70700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ba0106630 0x7f8ba01a4b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800b970 tx=0x7f8b9800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.524+0000 7f8b9d7fa700 1 -- 172.21.15.49:0/2749321588 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9800d860 con 0x7f8ba0106630 2024-03-26T18:00:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.524+0000 7f8b9d7fa700 1 -- 172.21.15.49:0/2749321588 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b9800e430 con 0x7f8ba0106630 2024-03-26T18:00:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.524+0000 7f8b9d7fa700 1 -- 172.21.15.49:0/2749321588 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98017660 con 0x7f8ba0106630 2024-03-26T18:00:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.524+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba01a52a0 con 0x7f8ba0106630 2024-03-26T18:00:37.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.524+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba01a5760 con 0x7f8ba0106630 2024-03-26T18:00:37.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.525+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ba0062ac0 con 0x7f8ba0106630 2024-03-26T18:00:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.526+0000 7f8b9d7fa700 1 -- 172.21.15.49:0/2749321588 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f8b9800e5e0 con 0x7f8ba0106630 2024-03-26T18:00:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.527+0000 7f8b9d7fa700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8b880a3040 0x7f8b880a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.527+0000 7f8b9d7fa700 1 -- 172.21.15.49:0/2749321588 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8b980cf7a0 con 0x7f8ba0106630 2024-03-26T18:00:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.527+0000 7f8b9ffff700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8b880a3040 0x7f8b880a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.527+0000 7f8b9d7fa700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] conn(0x7f8b880add00 0x7f8b880b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.527+0000 7f8b9d7fa700 1 -- 172.21.15.49:0/2749321588 --> [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b98011ec0 con 0x7f8b880add00 2024-03-26T18:00:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.527+0000 7f8b9d7fa700 1 -- 172.21.15.49:0/2749321588 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8b980cfbf0 con 0x7f8ba0106630 2024-03-26T18:00:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.527+0000 7f8b9f7fe700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] conn(0x7f8b880add00 0x7f8b880b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.528+0000 7f8b9ffff700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8b880a3040 0x7f8b880a5500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8b90005950 tx=0x7f8b9000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:37.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.528+0000 7f8b9f7fe700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] conn(0x7f8b880add00 0x7f8b880b0140 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:37.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.529+0000 7f8b9d7fa700 1 -- 172.21.15.49:0/2749321588 <== osd.7 v2:172.21.15.115:6816/32789491 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b98011ec0 con 0x7f8b880add00 2024-03-26T18:00:37.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.545+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 --> [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ba0062ec0 con 0x7f8b880add00 2024-03-26T18:00:37.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8b9d7fa700 1 -- 172.21.15.49:0/2749321588 <== osd.7 v2:172.21.15.115:6816/32789491 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8ba0062ec0 con 0x7f8b880add00 2024-03-26T18:00:37.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 >> [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] conn(0x7f8b880add00 msgr2=0x7f8b880b0140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:37.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba67d3700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] conn(0x7f8b880add00 0x7f8b880b0140 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:37.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8b880a3040 msgr2=0x7f8b880a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:37.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba67d3700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8b880a3040 0x7f8b880a5500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8b90005950 tx=0x7f8b9000b410 comp rx=0 tx=0).stop 2024-03-26T18:00:37.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ba0106630 msgr2=0x7f8ba01a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:37.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba67d3700 1 --2- 172.21.15.49:0/2749321588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ba0106630 0x7f8ba01a4b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800b970 tx=0x7f8b9800bc80 comp rx=0 tx=0).stop 2024-03-26T18:00:37.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba4d70700 1 -- 172.21.15.49:0/2749321588 reap_dead start 2024-03-26T18:00:37.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 shutdown_connections 2024-03-26T18:00:37.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 >> 172.21.15.49:0/2749321588 conn(0x7f8ba00778b0 msgr2=0x7f8ba00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:37.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 shutdown_connections 2024-03-26T18:00:37.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:37.546+0000 7f8ba67d3700 1 -- 172.21.15.49:0/2749321588 wait complete. 2024-03-26T18:00:38.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.101+0000 7f3aa28b1700 1 -- 172.21.15.49:0/3060564952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c101c50 msgr2=0x7f3a9c10e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.101+0000 7f3aa28b1700 1 --2- 172.21.15.49:0/3060564952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c101c50 0x7f3a9c10e130 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400b0a0 tx=0x7f3a9400b3b0 comp rx=0 tx=0).stop 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.101+0000 7f3aa28b1700 1 -- 172.21.15.49:0/3060564952 shutdown_connections 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.101+0000 7f3aa28b1700 1 --2- 172.21.15.49:0/3060564952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c101c50 0x7f3a9c10e130 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.101+0000 7f3aa28b1700 1 --2- 172.21.15.49:0/3060564952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a9c1072f0 0x7f3a9c1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.101+0000 7f3aa28b1700 1 --2- 172.21.15.49:0/3060564952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c1012b0 0x7f3a9c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.101+0000 7f3aa28b1700 1 -- 172.21.15.49:0/3060564952 >> 172.21.15.49:0/3060564952 conn(0x7f3a9c0781e0 msgr2=0x7f3a9c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.102+0000 7f3aa28b1700 1 -- 172.21.15.49:0/3060564952 shutdown_connections 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.102+0000 7f3aa28b1700 1 -- 172.21.15.49:0/3060564952 wait complete. 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.102+0000 7f3aa28b1700 1 Processor -- start 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.102+0000 7f3aa28b1700 1 -- start start 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3aa28b1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a9c1012b0 0x7f3a9c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3aa28b1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c101c50 0x7f3a9c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3aa28b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c1072f0 0x7f3a9c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3aa28b1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c111540 con 0x7f3a9c101c50 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3aa28b1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1113c0 con 0x7f3a9c1072f0 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3aa28b1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1116c0 con 0x7f3a9c1012b0 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a9b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c101c50 0x7f3a9c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a9b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c101c50 0x7f3a9c19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58116/0 (socket says 172.21.15.49:58116) 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a9bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a9c1012b0 0x7f3a9c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a9b7fe700 1 -- 172.21.15.49:0/688712130 learned_addr learned my addr 172.21.15.49:0/688712130 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3aa0e4e700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c1072f0 0x7f3a9c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a9b7fe700 1 -- 172.21.15.49:0/688712130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a9c1012b0 msgr2=0x7f3a9c19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a9b7fe700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a9c1012b0 0x7f3a9c19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a9b7fe700 1 -- 172.21.15.49:0/688712130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c1072f0 msgr2=0x7f3a9c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a9b7fe700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c1072f0 0x7f3a9c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a9b7fe700 1 -- 172.21.15.49:0/688712130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c199040 con 0x7f3a9c101c50 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a9b7fe700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c101c50 0x7f3a9c19ec60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400e9a0 tx=0x7f3a8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a997fa700 1 -- 172.21.15.49:0/688712130 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400ca10 con 0x7f3a9c101c50 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a997fa700 1 -- 172.21.15.49:0/688712130 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8400cbb0 con 0x7f3a9c101c50 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9c199330 con 0x7f3a9c101c50 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3a997fa700 1 -- 172.21.15.49:0/688712130 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84018680 con 0x7f3a9c101c50 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.103+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9c1a55f0 con 0x7f3a9c101c50 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.104+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a9c1053b0 con 0x7f3a9c101c50 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.106+0000 7f3a997fa700 1 -- 172.21.15.49:0/688712130 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f3a84010430 con 0x7f3a9c101c50 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.106+0000 7f3a997fa700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3a880a3040 0x7f3a880a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.106+0000 7f3a9bfff700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3a880a3040 0x7f3a880a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.106+0000 7f3a997fa700 1 -- 172.21.15.49:0/688712130 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3a840d0210 con 0x7f3a9c101c50 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.106+0000 7f3a997fa700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] conn(0x7f3a880add00 0x7f3a880b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.107+0000 7f3a997fa700 1 -- 172.21.15.49:0/688712130 --> [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a8401bec0 con 0x7f3a880add00 2024-03-26T18:00:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.107+0000 7f3a997fa700 1 -- 172.21.15.49:0/688712130 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3a840d0660 con 0x7f3a9c101c50 2024-03-26T18:00:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.107+0000 7f3aa0e4e700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] conn(0x7f3a880add00 0x7f3a880b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.107+0000 7f3a9bfff700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3a880a3040 0x7f3a880a5500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c199a20 tx=0x7f3a8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.107+0000 7f3aa0e4e700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] conn(0x7f3a880add00 0x7f3a880b0140 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.108+0000 7f3a997fa700 1 -- 172.21.15.49:0/688712130 <== osd.1 v2:172.21.15.115:6800/1211178238 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a8401bec0 con 0x7f3a880add00 2024-03-26T18:00:38.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.125+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 --> [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a9c1029a0 con 0x7f3a880add00 2024-03-26T18:00:38.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3a997fa700 1 -- 172.21.15.49:0/688712130 <== osd.1 v2:172.21.15.115:6800/1211178238 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3a9c1029a0 con 0x7f3a880add00 2024-03-26T18:00:38.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 >> [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] conn(0x7f3a880add00 msgr2=0x7f3a880b0140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:38.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3aa28b1700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] conn(0x7f3a880add00 0x7f3a880b0140 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:38.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3a880a3040 msgr2=0x7f3a880a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:38.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3aa28b1700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3a880a3040 0x7f3a880a5500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c199a20 tx=0x7f3a8c009260 comp rx=0 tx=0).stop 2024-03-26T18:00:38.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c101c50 msgr2=0x7f3a9c19ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:38.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3aa28b1700 1 --2- 172.21.15.49:0/688712130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a9c101c50 0x7f3a9c19ec60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400e9a0 tx=0x7f3a8400ecb0 comp rx=0 tx=0).stop 2024-03-26T18:00:38.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3aa0e4e700 1 -- 172.21.15.49:0/688712130 reap_dead start 2024-03-26T18:00:38.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 shutdown_connections 2024-03-26T18:00:38.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 >> 172.21.15.49:0/688712130 conn(0x7f3a9c0781e0 msgr2=0x7f3a9c100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:38.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.126+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 shutdown_connections 2024-03-26T18:00:38.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:38.127+0000 7f3aa28b1700 1 -- 172.21.15.49:0/688712130 wait complete. 2024-03-26T18:00:38.544 INFO:teuthology.orchestra.run.smithi049.stdout:111669149758 2024-03-26T18:00:38.545 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-26T18:00:38.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:38 smithi049 ceph-mon[32331]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-26T18:00:38.801 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:38 smithi115 ceph-mon[36174]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-26T18:00:38.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:38 smithi171 ceph-mon[36033]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-26T18:00:39.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.032+0000 7f258895b700 1 -- 172.21.15.49:0/2009519343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25841083e0 msgr2=0x7f25841087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.032+0000 7f258895b700 1 --2- 172.21.15.49:0/2009519343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25841083e0 0x7f25841087c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f25740099c0 tx=0x7f2574009cd0 comp rx=0 tx=0).stop 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.032+0000 7f258895b700 1 -- 172.21.15.49:0/2009519343 shutdown_connections 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.032+0000 7f258895b700 1 --2- 172.21.15.49:0/2009519343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2584102dc0 0x7f258410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.032+0000 7f258895b700 1 --2- 172.21.15.49:0/2009519343 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25841083e0 0x7f25841087c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.032+0000 7f258895b700 1 --2- 172.21.15.49:0/2009519343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2584102400 0x7f2584102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.032+0000 7f258895b700 1 -- 172.21.15.49:0/2009519343 >> 172.21.15.49:0/2009519343 conn(0x7f25840794e0 msgr2=0x7f25840798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.032+0000 7f258895b700 1 -- 172.21.15.49:0/2009519343 shutdown_connections 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.032+0000 7f258895b700 1 -- 172.21.15.49:0/2009519343 wait complete. 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.032+0000 7f258895b700 1 Processor -- start 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f258895b700 1 -- start start 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f258895b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2584102400 0x7f25840746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:39.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f258895b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2584102dc0 0x7f2584074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f258895b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25841083e0 0x7f2584075130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f258895b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2584112690 con 0x7f25841083e0 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f258895b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2584112510 con 0x7f2584102dc0 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f258895b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2584112810 con 0x7f2584102400 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f2582d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25841083e0 0x7f2584075130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f258259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2584102400 0x7f25840746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f2581d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2584102dc0 0x7f2584074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f258259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2584102400 0x7f25840746b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:35090/0 (socket says 172.21.15.49:35090) 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f2582d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25841083e0 0x7f2584075130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58106/0 (socket says 172.21.15.49:58106) 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f258259c700 1 -- 172.21.15.49:0/624030104 learned_addr learned my addr 172.21.15.49:0/624030104 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f2581d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2584102dc0 0x7f2584074bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48934/0 (socket says 172.21.15.49:48934) 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f2581d9b700 1 -- 172.21.15.49:0/624030104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2584102400 msgr2=0x7f25840746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f2581d9b700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2584102400 0x7f25840746b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:39.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f2581d9b700 1 -- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25841083e0 msgr2=0x7f2584075130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:39.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f2581d9b700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f25841083e0 0x7f2584075130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:39.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f2581d9b700 1 -- 172.21.15.49:0/624030104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25841a6fd0 con 0x7f2584102dc0 2024-03-26T18:00:39.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.033+0000 7f2581d9b700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2584102dc0 0x7f2584074bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f257800e9f0 tx=0x7f257800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:39.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.034+0000 7f25737fe700 1 -- 172.21.15.49:0/624030104 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578004220 con 0x7f2584102dc0 2024-03-26T18:00:39.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.034+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25841a72c0 con 0x7f2584102dc0 2024-03-26T18:00:39.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.034+0000 7f25737fe700 1 -- 172.21.15.49:0/624030104 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2578004d10 con 0x7f2584102dc0 2024-03-26T18:00:39.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.034+0000 7f25737fe700 1 -- 172.21.15.49:0/624030104 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578005020 con 0x7f2584102dc0 2024-03-26T18:00:39.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.034+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25841a77b0 con 0x7f2584102dc0 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.034+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2584062b00 con 0x7f2584102dc0 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.036+0000 7f25737fe700 1 -- 172.21.15.49:0/624030104 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f2578004750 con 0x7f2584102dc0 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.036+0000 7f25737fe700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f256c0a3040 0x7f256c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.037+0000 7f258259c700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f256c0a3040 0x7f256c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.037+0000 7f25737fe700 1 -- 172.21.15.49:0/624030104 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f25780cf970 con 0x7f2584102dc0 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.037+0000 7f25737fe700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] conn(0x7f256c0add00 0x7f256c0b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.037+0000 7f25737fe700 1 -- 172.21.15.49:0/624030104 --> [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2578011e50 con 0x7f256c0add00 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.037+0000 7f25737fe700 1 -- 172.21.15.49:0/624030104 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f25780cfd20 con 0x7f2584102dc0 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.037+0000 7f2582d9d700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] conn(0x7f256c0add00 0x7f256c0b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.037+0000 7f258259c700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f256c0a3040 0x7f256c0a5500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2574009750 tx=0x7f2574005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.037+0000 7f2582d9d700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] conn(0x7f256c0add00 0x7f256c0b0140 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.038+0000 7f25737fe700 1 -- 172.21.15.49:0/624030104 <== osd.9 v2:172.21.15.49:6826/1156582500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2578011e50 con 0x7f256c0add00 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.054+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 --> [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2584062f80 con 0x7f256c0add00 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.054+0000 7f25737fe700 1 -- 172.21.15.49:0/624030104 <== osd.9 v2:172.21.15.49:6826/1156582500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2584062f80 con 0x7f256c0add00 2024-03-26T18:00:39.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.054+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] conn(0x7f256c0add00 msgr2=0x7f256c0b0140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:39.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.055+0000 7f258895b700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] conn(0x7f256c0add00 0x7f256c0b0140 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:39.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.055+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f256c0a3040 msgr2=0x7f256c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:39.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.055+0000 7f258895b700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f256c0a3040 0x7f256c0a5500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2574009750 tx=0x7f2574005960 comp rx=0 tx=0).stop 2024-03-26T18:00:39.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.055+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2584102dc0 msgr2=0x7f2584074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:39.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.055+0000 7f258895b700 1 --2- 172.21.15.49:0/624030104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2584102dc0 0x7f2584074bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f257800e9f0 tx=0x7f257800ed00 comp rx=0 tx=0).stop 2024-03-26T18:00:39.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.055+0000 7f2582d9d700 1 -- 172.21.15.49:0/624030104 reap_dead start 2024-03-26T18:00:39.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.055+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 shutdown_connections 2024-03-26T18:00:39.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.055+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 >> 172.21.15.49:0/624030104 conn(0x7f25840794e0 msgr2=0x7f2584100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:39.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.055+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 shutdown_connections 2024-03-26T18:00:39.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.055+0000 7f258895b700 1 -- 172.21.15.49:0/624030104 wait complete. 2024-03-26T18:00:40.021 INFO:teuthology.orchestra.run.smithi049.stdout:223338299435 2024-03-26T18:00:40.021 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-26T18:00:40.461 INFO:teuthology.orchestra.run.smithi049.stdout:292057776162 2024-03-26T18:00:40.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-26T18:00:40.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:40 smithi049 ceph-mon[32331]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-26T18:00:40.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:40 smithi171 ceph-mon[36033]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-26T18:00:40.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.986+0000 7fc772222700 1 -- 172.21.15.49:0/1435832804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76c1076f0 msgr2=0x7fc76c10f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:40.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.986+0000 7fc772222700 1 --2- 172.21.15.49:0/1435832804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76c1076f0 0x7fc76c10f2a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc76400b0a0 tx=0x7fc76400b3b0 comp rx=0 tx=0).stop 2024-03-26T18:00:40.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.986+0000 7fc772222700 1 -- 172.21.15.49:0/1435832804 shutdown_connections 2024-03-26T18:00:40.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.986+0000 7fc772222700 1 --2- 172.21.15.49:0/1435832804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76c1076f0 0x7fc76c10f2a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:40.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.986+0000 7fc772222700 1 --2- 172.21.15.49:0/1435832804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c06ad10 0x7fc76c06b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:40.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.986+0000 7fc772222700 1 --2- 172.21.15.49:0/1435832804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c06b6c0 0x7fc76c107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:40.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.986+0000 7fc772222700 1 -- 172.21.15.49:0/1435832804 >> 172.21.15.49:0/1435832804 conn(0x7fc76c077300 msgr2=0x7fc76c077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:40.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.986+0000 7fc772222700 1 -- 172.21.15.49:0/1435832804 shutdown_connections 2024-03-26T18:00:40.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.986+0000 7fc772222700 1 -- 172.21.15.49:0/1435832804 wait complete. 2024-03-26T18:00:40.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.987+0000 7fc772222700 1 Processor -- start 2024-03-26T18:00:40.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.987+0000 7fc772222700 1 -- start start 2024-03-26T18:00:40.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.987+0000 7fc772222700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76c06ad10 0x7fc76c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:40.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.987+0000 7fc772222700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c06b6c0 0x7fc76c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:40.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.987+0000 7fc772222700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c1076f0 0x7fc76c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:40.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.987+0000 7fc772222700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1126b0 con 0x7fc76c06b6c0 2024-03-26T18:00:40.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.987+0000 7fc772222700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c112530 con 0x7fc76c06ad10 2024-03-26T18:00:40.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.987+0000 7fc772222700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c112830 con 0x7fc76c1076f0 2024-03-26T18:00:40.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.987+0000 7fc76affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c06b6c0 0x7fc76c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:40.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76c06ad10 0x7fc76c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:40.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76c06ad10 0x7fc76c19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48954/0 (socket says 172.21.15.49:48954) 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76b7fe700 1 -- 172.21.15.49:0/467726297 learned_addr learned my addr 172.21.15.49:0/467726297 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c06b6c0 0x7fc76c19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58162/0 (socket says 172.21.15.49:58162) 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76bfff700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c1076f0 0x7fc76c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76b7fe700 1 -- 172.21.15.49:0/467726297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c1076f0 msgr2=0x7fc76c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76b7fe700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c1076f0 0x7fc76c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76b7fe700 1 -- 172.21.15.49:0/467726297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c06b6c0 msgr2=0x7fc76c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76b7fe700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c06b6c0 0x7fc76c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76b7fe700 1 -- 172.21.15.49:0/467726297 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c199030 con 0x7fc76c06ad10 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc76b7fe700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76c06ad10 0x7fc76c19e680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9c0 tx=0x7fc75c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc768ff9700 1 -- 172.21.15.49:0/467726297 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00cbd0 con 0x7fc76c06ad10 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc768ff9700 1 -- 172.21.15.49:0/467726297 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75c00ee60 con 0x7fc76c06ad10 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c199320 con 0x7fc76c06ad10 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc768ff9700 1 -- 172.21.15.49:0/467726297 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c005630 con 0x7fc76c06ad10 2024-03-26T18:00:40.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.988+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c1a54e0 con 0x7fc76c06ad10 2024-03-26T18:00:40.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.989+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc76c0ffc80 con 0x7fc76c06ad10 2024-03-26T18:00:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.990+0000 7fc768ff9700 1 -- 172.21.15.49:0/467726297 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fc75c044b80 con 0x7fc76c06ad10 2024-03-26T18:00:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.991+0000 7fc768ff9700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7540a3040 0x7fc7540a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.991+0000 7fc76affd700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7540a3040 0x7fc7540a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.991+0000 7fc768ff9700 1 -- 172.21.15.49:0/467726297 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc75c0cfbb0 con 0x7fc76c06ad10 2024-03-26T18:00:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.991+0000 7fc768ff9700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] conn(0x7fc7540add00 0x7fc7540b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.991+0000 7fc768ff9700 1 -- 172.21.15.49:0/467726297 --> [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7540b0810 con 0x7fc7540add00 2024-03-26T18:00:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.991+0000 7fc768ff9700 1 -- 172.21.15.49:0/467726297 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc75c0d0000 con 0x7fc76c06ad10 2024-03-26T18:00:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.991+0000 7fc76bfff700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] conn(0x7fc7540add00 0x7fc7540b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.991+0000 7fc76affd700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7540a3040 0x7fc7540a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc760009680 tx=0x7fc760009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:40.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.992+0000 7fc76bfff700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] conn(0x7fc7540add00 0x7fc7540b0140 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:40.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:40.993+0000 7fc768ff9700 1 -- 172.21.15.49:0/467726297 <== osd.8 v2:172.21.15.171:6816/4095981725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7540b0810 con 0x7fc7540add00 2024-03-26T18:00:40.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:40 smithi115 ceph-mon[36174]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-26T18:00:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.009+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 --> [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc76c103af0 con 0x7fc7540add00 2024-03-26T18:00:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.009+0000 7fc768ff9700 1 -- 172.21.15.49:0/467726297 <== osd.8 v2:172.21.15.171:6816/4095981725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc76c103af0 con 0x7fc7540add00 2024-03-26T18:00:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 >> [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] conn(0x7fc7540add00 msgr2=0x7fc7540b0140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc772222700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] conn(0x7fc7540add00 0x7fc7540b0140 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7540a3040 msgr2=0x7fc7540a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc772222700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc7540a3040 0x7fc7540a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc760009680 tx=0x7fc760009260 comp rx=0 tx=0).stop 2024-03-26T18:00:40.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76c06ad10 msgr2=0x7fc76c19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc772222700 1 --2- 172.21.15.49:0/467726297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76c06ad10 0x7fc76c19e680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9c0 tx=0x7fc75c00ecd0 comp rx=0 tx=0).stop 2024-03-26T18:00:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc76bfff700 1 -- 172.21.15.49:0/467726297 reap_dead start 2024-03-26T18:00:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 shutdown_connections 2024-03-26T18:00:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 >> 172.21.15.49:0/467726297 conn(0x7fc76c077300 msgr2=0x7fc76c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 shutdown_connections 2024-03-26T18:00:40.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:41.010+0000 7fc772222700 1 -- 172.21.15.49:0/467726297 wait complete. 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.164+0000 7f8204df2700 1 -- 172.21.15.49:0/1669317949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f820006ab50 msgr2=0x7f820006af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.164+0000 7f8204df2700 1 --2- 172.21.15.49:0/1669317949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f820006ab50 0x7f820006af30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f81f00099c0 tx=0x7f81f0009cd0 comp rx=0 tx=0).stop 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.164+0000 7f8204df2700 1 -- 172.21.15.49:0/1669317949 shutdown_connections 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.164+0000 7f8204df2700 1 --2- 172.21.15.49:0/1669317949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8200107730 0x7f820010f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.164+0000 7f8204df2700 1 --2- 172.21.15.49:0/1669317949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f820006ab50 0x7f820006af30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.164+0000 7f8204df2700 1 --2- 172.21.15.49:0/1669317949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f820006b500 0x7f8200107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.164+0000 7f8204df2700 1 -- 172.21.15.49:0/1669317949 >> 172.21.15.49:0/1669317949 conn(0x7f8200077300 msgr2=0x7f8200077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.164+0000 7f8204df2700 1 -- 172.21.15.49:0/1669317949 shutdown_connections 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.164+0000 7f8204df2700 1 -- 172.21.15.49:0/1669317949 wait complete. 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f8204df2700 1 Processor -- start 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f8204df2700 1 -- start start 2024-03-26T18:00:41.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f8204df2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f820006ab50 0x7f820019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f8204df2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f820006b500 0x7f820019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f8204df2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8200107730 0x7f8200198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f8204df2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f82001126c0 con 0x7f820006ab50 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f8204df2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8200112540 con 0x7f820006b500 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f8204df2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8200112840 con 0x7f8200107730 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f81fe59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f820006ab50 0x7f820019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f81fe59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f820006ab50 0x7f820019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58134/0 (socket says 172.21.15.49:58134) 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f81fe59c700 1 -- 172.21.15.49:0/2456815416 learned_addr learned my addr 172.21.15.49:0/2456815416 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f81fdd9b700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f820006b500 0x7f820019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.165+0000 7f81fed9d700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8200107730 0x7f8200198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f81fe59c700 1 -- 172.21.15.49:0/2456815416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8200107730 msgr2=0x7f8200198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f81fe59c700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8200107730 0x7f8200198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f81fe59c700 1 -- 172.21.15.49:0/2456815416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f820006b500 msgr2=0x7f820019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f81fe59c700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f820006b500 0x7f820019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:41.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f81fe59c700 1 -- 172.21.15.49:0/2456815416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8200199160 con 0x7f820006ab50 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f81fe59c700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f820006ab50 0x7f820019e840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f81f0006010 tx=0x7f81f0003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f81ef7fe700 1 -- 172.21.15.49:0/2456815416 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0003680 con 0x7f820006ab50 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f81ef7fe700 1 -- 172.21.15.49:0/2456815416 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81f0003e50 con 0x7f820006ab50 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82001993f0 con 0x7f820006ab50 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f81ef7fe700 1 -- 172.21.15.49:0/2456815416 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f000fe10 con 0x7f820006ab50 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.166+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82001a5710 con 0x7f820006ab50 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.167+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8200062ac0 con 0x7f820006ab50 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.168+0000 7f81ef7fe700 1 -- 172.21.15.49:0/2456815416 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f81f000f460 con 0x7f820006ab50 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.169+0000 7f81ef7fe700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f81e80a31e0 0x7f81e80a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.169+0000 7f81fdd9b700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f81e80a31e0 0x7f81e80a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.169+0000 7f81ef7fe700 1 -- 172.21.15.49:0/2456815416 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f81f00d0670 con 0x7f820006ab50 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.169+0000 7f81ef7fe700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] conn(0x7f81e80add70 0x7f81e80b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.169+0000 7f81ef7fe700 1 -- 172.21.15.49:0/2456815416 --> [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81f0001d30 con 0x7f81e80add70 2024-03-26T18:00:41.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.169+0000 7f81ef7fe700 1 -- 172.21.15.49:0/2456815416 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f81f00d0ac0 con 0x7f820006ab50 2024-03-26T18:00:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.169+0000 7f81fed9d700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] conn(0x7f81e80add70 0x7f81e80b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.169+0000 7f81fdd9b700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f81e80a31e0 0x7f81e80a56a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009680 tx=0x7f81f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.170+0000 7f81fed9d700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] conn(0x7f81e80add70 0x7f81e80b01b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.170+0000 7f81ef7fe700 1 -- 172.21.15.49:0/2456815416 <== osd.0 v2:172.21.15.49:6802/1257601781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81f0001d30 con 0x7f81e80add70 2024-03-26T18:00:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.186+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 --> [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8200062f40 con 0x7f81e80add70 2024-03-26T18:00:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f81ef7fe700 1 -- 172.21.15.49:0/2456815416 <== osd.0 v2:172.21.15.49:6802/1257601781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8200062f40 con 0x7f81e80add70 2024-03-26T18:00:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] conn(0x7f81e80add70 msgr2=0x7f81e80b01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f8204df2700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] conn(0x7f81e80add70 0x7f81e80b01b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f81e80a31e0 msgr2=0x7f81e80a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f8204df2700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f81e80a31e0 0x7f81e80a56a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009680 tx=0x7f81f4009260 comp rx=0 tx=0).stop 2024-03-26T18:00:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f820006ab50 msgr2=0x7f820019e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f8204df2700 1 --2- 172.21.15.49:0/2456815416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f820006ab50 0x7f820019e840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f81f0006010 tx=0x7f81f0003a90 comp rx=0 tx=0).stop 2024-03-26T18:00:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f81fed9d700 1 -- 172.21.15.49:0/2456815416 reap_dead start 2024-03-26T18:00:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 shutdown_connections 2024-03-26T18:00:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 >> 172.21.15.49:0/2456815416 conn(0x7f8200077300 msgr2=0x7f82000ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 shutdown_connections 2024-03-26T18:00:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:39.187+0000 7f8204df2700 1 -- 172.21.15.49:0/2456815416 wait complete. 2024-03-26T18:00:41.625 INFO:teuthology.orchestra.run.smithi049.stdout:34359738446 2024-03-26T18:00:41.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-26T18:00:42.561 INFO:teuthology.orchestra.run.smithi049.stdout:347892350998 2024-03-26T18:00:42.562 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-26T18:00:42.837 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:42 smithi049 ceph-mon[32331]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T18:00:42.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:42 smithi171 ceph-mon[36033]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T18:00:42.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:42 smithi115 ceph-mon[36174]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T18:00:43.165 INFO:teuthology.orchestra.run.smithi049.stdout:257698037797 2024-03-26T18:00:43.165 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-26T18:00:43.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-26T18:00:43.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-26T18:00:43.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]: dispatch 2024-03-26T18:00:43.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-26T18:00:43.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-03-26T18:00:43.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-26T18:00:43.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-03-26T18:00:43.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-03-26T18:00:43.753 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:43 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-26T18:00:43.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-26T18:00:43.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-26T18:00:43.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]: dispatch 2024-03-26T18:00:43.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-26T18:00:43.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-03-26T18:00:43.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-26T18:00:43.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-03-26T18:00:43.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-03-26T18:00:43.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:43 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-26T18:00:43.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-26T18:00:43.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-26T18:00:43.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]: dispatch 2024-03-26T18:00:43.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-26T18:00:43.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2024-03-26T18:00:43.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-26T18:00:43.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-03-26T18:00:43.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-03-26T18:00:43.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:43 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-26T18:00:44.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-26T18:00:44.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-26T18:00:44.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-26T18:00:44.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-03-26T18:00:44.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-26T18:00:44.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-03-26T18:00:44.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-26T18:00:44.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-03-26T18:00:44.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-03-26T18:00:44.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-26T18:00:44.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:44 smithi171 ceph-mon[36033]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T18:00:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-26T18:00:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-26T18:00:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-26T18:00:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-03-26T18:00:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-26T18:00:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-03-26T18:00:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-26T18:00:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-03-26T18:00:44.919 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-03-26T18:00:44.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-26T18:00:44.920 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:44 smithi049 ceph-mon[32331]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T18:00:44.956 INFO:teuthology.orchestra.run.smithi049.stdout:55834574922 2024-03-26T18:00:44.957 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-26T18:00:44.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-26T18:00:44.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-26T18:00:44.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-26T18:00:44.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-03-26T18:00:44.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-26T18:00:44.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2024-03-26T18:00:44.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-26T18:00:44.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-03-26T18:00:44.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-03-26T18:00:44.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-26T18:00:44.988 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:44 smithi115 ceph-mon[36174]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T18:00:45.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:45 smithi049 ceph-mon[32331]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T18:00:45.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:45 smithi171 ceph-mon[36033]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T18:00:45.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:45 smithi115 ceph-mon[36174]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T18:00:46.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.199+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/2473418220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a810a650 msgr2=0x7fb2a810aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.199+0000 7fb2ae5c0700 1 --2- 172.21.15.49:0/2473418220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a810a650 0x7fb2a810aa30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb2980099c0 tx=0x7fb298009cd0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.199+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/2473418220 shutdown_connections 2024-03-26T18:00:46.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.199+0000 7fb2ae5c0700 1 --2- 172.21.15.49:0/2473418220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a8104f60 0x7fb2a8111420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.199+0000 7fb2ae5c0700 1 --2- 172.21.15.49:0/2473418220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a810a650 0x7fb2a810aa30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.199+0000 7fb2ae5c0700 1 --2- 172.21.15.49:0/2473418220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a81045a0 0x7fb2a8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.199+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/2473418220 >> 172.21.15.49:0/2473418220 conn(0x7fb2a8100090 msgr2=0x7fb2a81024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:46.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.199+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/2473418220 shutdown_connections 2024-03-26T18:00:46.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.199+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/2473418220 wait complete. 2024-03-26T18:00:46.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2ae5c0700 1 Processor -- start 2024-03-26T18:00:46.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2ae5c0700 1 -- start start 2024-03-26T18:00:46.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2ae5c0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a81045a0 0x7fb2a81a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2ae5c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8104f60 0x7fb2a81a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2ae5c0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a810a650 0x7fb2a819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2ae5c0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81148b0 con 0x7fb2a8104f60 2024-03-26T18:00:46.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2ae5c0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a8114730 con 0x7fb2a81045a0 2024-03-26T18:00:46.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2ae5c0700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a8114a30 con 0x7fb2a810a650 2024-03-26T18:00:46.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2a77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8104f60 0x7fb2a81a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2a77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8104f60 0x7fb2a81a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58184/0 (socket says 172.21.15.49:58184) 2024-03-26T18:00:46.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a81045a0 0x7fb2a81a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2a77fe700 1 -- 172.21.15.49:0/1658259174 learned_addr learned my addr 172.21.15.49:0/1658259174 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:46.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.200+0000 7fb2acb5d700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a810a650 0x7fb2a819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2a77fe700 1 -- 172.21.15.49:0/1658259174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a810a650 msgr2=0x7fb2a819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2a77fe700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a810a650 0x7fb2a819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2a77fe700 1 -- 172.21.15.49:0/1658259174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a81045a0 msgr2=0x7fb2a81a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2a77fe700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a81045a0 0x7fb2a81a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2a77fe700 1 -- 172.21.15.49:0/1658259174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a819b230 con 0x7fb2a8104f60 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2acb5d700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2a810a650 0x7fb2a819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2a7fff700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a81045a0 0x7fb2a81a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2a77fe700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8104f60 0x7fb2a81a0e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00d950 tx=0x7fb29c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2a57fa700 1 -- 172.21.15.49:0/1658259174 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c016070 con 0x7fb2a8104f60 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2a57fa700 1 -- 172.21.15.49:0/1658259174 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29c00fd70 con 0x7fb2a8104f60 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a819b520 con 0x7fb2a8104f60 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2a57fa700 1 -- 172.21.15.49:0/1658259174 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00e610 con 0x7fb2a8104f60 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.201+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a81a7790 con 0x7fb2a8104f60 2024-03-26T18:00:46.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.202+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb2a8062b40 con 0x7fb2a8104f60 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.203+0000 7fb2a57fa700 1 -- 172.21.15.49:0/1658259174 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fb29c019070 con 0x7fb2a8104f60 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.204+0000 7fb2a57fa700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb2900a3040 0x7fb2900a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.204+0000 7fb2a7fff700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb2900a3040 0x7fb2900a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.204+0000 7fb2a57fa700 1 -- 172.21.15.49:0/1658259174 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fb29c0cf900 con 0x7fb2a8104f60 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.204+0000 7fb2a57fa700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] conn(0x7fb2900ae120 0x7fb2900b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.204+0000 7fb2a57fa700 1 -- 172.21.15.49:0/1658259174 --> [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb29c005690 con 0x7fb2900ae120 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.204+0000 7fb2a57fa700 1 -- 172.21.15.49:0/1658259174 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb29c0cfcb0 con 0x7fb2a8104f60 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.204+0000 7fb2a7fff700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb2900a3040 0x7fb2900a5500 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb298000c00 tx=0x7fb298005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.204+0000 7fb2acb5d700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] conn(0x7fb2900ae120 0x7fb2900b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.205+0000 7fb2acb5d700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] conn(0x7fb2900ae120 0x7fb2900b05c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.206+0000 7fb2a57fa700 1 -- 172.21.15.49:0/1658259174 <== osd.5 v2:172.21.15.171:6808/2358624537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb29c005690 con 0x7fb2900ae120 2024-03-26T18:00:46.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.224+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 --> [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb2a8062fc0 con 0x7fb2900ae120 2024-03-26T18:00:46.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.225+0000 7fb2a57fa700 1 -- 172.21.15.49:0/1658259174 <== osd.5 v2:172.21.15.171:6808/2358624537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb2a8062fc0 con 0x7fb2900ae120 2024-03-26T18:00:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.225+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 >> [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] conn(0x7fb2900ae120 msgr2=0x7fb2900b05c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.225+0000 7fb2ae5c0700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] conn(0x7fb2900ae120 0x7fb2900b05c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.225+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb2900a3040 msgr2=0x7fb2900a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.225+0000 7fb2ae5c0700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb2900a3040 0x7fb2900a5500 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb298000c00 tx=0x7fb298005fb0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.225+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8104f60 msgr2=0x7fb2a81a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.225+0000 7fb2ae5c0700 1 --2- 172.21.15.49:0/1658259174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2a8104f60 0x7fb2a81a0e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00d950 tx=0x7fb29c00dc60 comp rx=0 tx=0).stop 2024-03-26T18:00:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.225+0000 7fb2acb5d700 1 -- 172.21.15.49:0/1658259174 reap_dead start 2024-03-26T18:00:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.226+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 shutdown_connections 2024-03-26T18:00:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.226+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 >> 172.21.15.49:0/1658259174 conn(0x7fb2a8100090 msgr2=0x7fb2a81016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:46.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.226+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 shutdown_connections 2024-03-26T18:00:46.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.226+0000 7fb2ae5c0700 1 -- 172.21.15.49:0/1658259174 wait complete. 2024-03-26T18:00:46.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.102+0000 7f9defb40700 1 -- 172.21.15.49:0/3988676485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8102df0 msgr2=0x7f9de810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.102+0000 7f9defb40700 1 --2- 172.21.15.49:0/3988676485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8102df0 0x7f9de810f250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9de000b0a0 tx=0x7f9de000b3b0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.102+0000 7f9defb40700 1 -- 172.21.15.49:0/3988676485 shutdown_connections 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.102+0000 7f9defb40700 1 --2- 172.21.15.49:0/3988676485 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8102df0 0x7f9de810f250 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.102+0000 7f9defb40700 1 --2- 172.21.15.49:0/3988676485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9de8108510 0x7f9de81088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.102+0000 7f9defb40700 1 --2- 172.21.15.49:0/3988676485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9de8102430 0x7f9de81028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.102+0000 7f9defb40700 1 -- 172.21.15.49:0/3988676485 >> 172.21.15.49:0/3988676485 conn(0x7f9de8079470 msgr2=0x7f9de8079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.102+0000 7f9defb40700 1 -- 172.21.15.49:0/3988676485 shutdown_connections 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.102+0000 7f9defb40700 1 -- 172.21.15.49:0/3988676485 wait complete. 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9defb40700 1 Processor -- start 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9defb40700 1 -- start start 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9defb40700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9de8102430 0x7f9de819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9defb40700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8102df0 0x7f9de819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9defb40700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9de8108510 0x7f9de8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9defb40700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8112690 con 0x7f9de8102df0 2024-03-26T18:00:46.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9defb40700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8112510 con 0x7f9de8108510 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9defb40700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8112810 con 0x7f9de8102430 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9ded0db700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8102df0 0x7f9de819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9ded0db700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8102df0 0x7f9de819ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58178/0 (socket says 172.21.15.49:58178) 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9ded0db700 1 -- 172.21.15.49:0/1197772605 learned_addr learned my addr 172.21.15.49:0/1197772605 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9ded8dc700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9de8102430 0x7f9de819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.103+0000 7f9dee0dd700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9de8108510 0x7f9de8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9ded0db700 1 -- 172.21.15.49:0/1197772605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9de8102430 msgr2=0x7f9de819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9ded0db700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9de8102430 0x7f9de819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9ded0db700 1 -- 172.21.15.49:0/1197772605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9de8108510 msgr2=0x7f9de8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9ded0db700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9de8108510 0x7f9de8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9ded0db700 1 -- 172.21.15.49:0/1197772605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de81990d0 con 0x7f9de8102df0 2024-03-26T18:00:46.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9dee0dd700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9de8108510 0x7f9de8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9ded0db700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8102df0 0x7f9de819ecf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9de400e9c0 tx=0x7f9de400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9ddeffd700 1 -- 172.21.15.49:0/1197772605 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de400cbd0 con 0x7f9de8102df0 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9ddeffd700 1 -- 172.21.15.49:0/1197772605 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de400ee60 con 0x7f9de8102df0 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de81993c0 con 0x7f9de8102df0 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9ddeffd700 1 -- 172.21.15.49:0/1197772605 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4005630 con 0x7f9de8102df0 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.104+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de81a5680 con 0x7f9de8102df0 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.105+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9de8062a80 con 0x7f9de8102df0 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.106+0000 7f9ddeffd700 1 -- 172.21.15.49:0/1197772605 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f9de4004750 con 0x7f9de8102df0 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.107+0000 7f9ddeffd700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9dd40a3040 0x7f9dd40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.107+0000 7f9ded8dc700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9dd40a3040 0x7f9dd40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.107+0000 7f9ddeffd700 1 -- 172.21.15.49:0/1197772605 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f9de40cff60 con 0x7f9de8102df0 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.107+0000 7f9ddeffd700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] conn(0x7f9dd40ae120 0x7f9dd40b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.107+0000 7f9ddeffd700 1 -- 172.21.15.49:0/1197772605 --> [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9de401ee50 con 0x7f9dd40ae120 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.107+0000 7f9ddeffd700 1 -- 172.21.15.49:0/1197772605 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f9de40d03b0 con 0x7f9de8102df0 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.107+0000 7f9dee0dd700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] conn(0x7f9dd40ae120 0x7f9dd40b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.108+0000 7f9ded8dc700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9dd40a3040 0x7f9dd40a5500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009680 tx=0x7f9dd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.108+0000 7f9dee0dd700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] conn(0x7f9dd40ae120 0x7f9dd40b05c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.109+0000 7f9ddeffd700 1 -- 172.21.15.49:0/1197772605 <== osd.6 v2:172.21.15.49:6818/2242936117 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9de401ee50 con 0x7f9dd40ae120 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.126+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 --> [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9de8062f00 con 0x7f9dd40ae120 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.126+0000 7f9ddeffd700 1 -- 172.21.15.49:0/1197772605 <== osd.6 v2:172.21.15.49:6818/2242936117 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9de8062f00 con 0x7f9dd40ae120 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.126+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] conn(0x7f9dd40ae120 msgr2=0x7f9dd40b05c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.126+0000 7f9defb40700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] conn(0x7f9dd40ae120 0x7f9dd40b05c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.126+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9dd40a3040 msgr2=0x7f9dd40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.126+0000 7f9defb40700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9dd40a3040 0x7f9dd40a5500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009680 tx=0x7f9dd8009260 comp rx=0 tx=0).stop 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.126+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8102df0 msgr2=0x7f9de819ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.126+0000 7f9defb40700 1 --2- 172.21.15.49:0/1197772605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9de8102df0 0x7f9de819ecf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9de400e9c0 tx=0x7f9de400ecd0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.126+0000 7f9dee0dd700 1 -- 172.21.15.49:0/1197772605 reap_dead start 2024-03-26T18:00:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.127+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 shutdown_connections 2024-03-26T18:00:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.127+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 >> 172.21.15.49:0/1197772605 conn(0x7f9de8079470 msgr2=0x7f9de8101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.127+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 shutdown_connections 2024-03-26T18:00:46.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:46.127+0000 7f9defb40700 1 -- 172.21.15.49:0/1197772605 wait complete. 2024-03-26T18:00:46.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:46 smithi171 ceph-mon[36033]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-26T18:00:46.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:46 smithi049 ceph-mon[32331]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-26T18:00:46.925 INFO:teuthology.orchestra.run.smithi049.stdout:201863462962 2024-03-26T18:00:46.925 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-26T18:00:46.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:46 smithi115 ceph-mon[36174]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-26T18:00:47.718 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:47 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:00:47.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:47 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:00:47.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:47 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:00:48.126 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:48.566 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:48.600 INFO:teuthology.orchestra.run.smithi049.stdout:167503724598 2024-03-26T18:00:48.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-26T18:00:48.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:48 smithi171 ceph-mon[36033]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-03-26T18:00:48.735 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:48 smithi115 ceph-mon[36174]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-03-26T18:00:48.828 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:48.840 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:48 smithi049 ceph-mon[32331]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-03-26T18:00:49.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 -- 172.21.15.49:0/2197201169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0c101540 msgr2=0x7f2f0c101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:49.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 --2- 172.21.15.49:0/2197201169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0c101540 0x7f2f0c101920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009990 tx=0x7f2efc009ca0 comp rx=0 tx=0).stop 2024-03-26T18:00:49.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 -- 172.21.15.49:0/2197201169 shutdown_connections 2024-03-26T18:00:49.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 --2- 172.21.15.49:0/2197201169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f0c1065d0 0x7f2f0c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:49.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 --2- 172.21.15.49:0/2197201169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0c101540 0x7f2f0c101920 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:49.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 --2- 172.21.15.49:0/2197201169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f0c101ef0 0x7f2f0c105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:49.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 -- 172.21.15.49:0/2197201169 >> 172.21.15.49:0/2197201169 conn(0x7f2f0c0772d0 msgr2=0x7f2f0c0776e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:49.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 -- 172.21.15.49:0/2197201169 shutdown_connections 2024-03-26T18:00:49.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 -- 172.21.15.49:0/2197201169 wait complete. 2024-03-26T18:00:49.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 Processor -- start 2024-03-26T18:00:49.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.498+0000 7f2f123c6700 1 -- start start 2024-03-26T18:00:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f123c6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0c101540 0x7f2f0c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f123c6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f0c101ef0 0x7f2f0c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f123c6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f0c1065d0 0x7f2f0c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f123c6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c1115b0 con 0x7f2f0c101540 2024-03-26T18:00:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f123c6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c111430 con 0x7f2f0c101ef0 2024-03-26T18:00:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f123c6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c111730 con 0x7f2f0c1065d0 2024-03-26T18:00:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f0bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0c101540 0x7f2f0c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f0bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0c101540 0x7f2f0c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53692/0 (socket says 172.21.15.49:53692) 2024-03-26T18:00:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f0bfff700 1 -- 172.21.15.49:0/3545654730 learned_addr learned my addr 172.21.15.49:0/3545654730 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:49.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f0b7fe700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f0c101ef0 0x7f2f0c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f10963700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f0c1065d0 0x7f2f0c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f0bfff700 1 -- 172.21.15.49:0/3545654730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f0c1065d0 msgr2=0x7f2f0c197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f0bfff700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f0c1065d0 0x7f2f0c197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f0bfff700 1 -- 172.21.15.49:0/3545654730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f0c101ef0 msgr2=0x7f2f0c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f0bfff700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f0c101ef0 0x7f2f0c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.499+0000 7f2f0bfff700 1 -- 172.21.15.49:0/3545654730 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c197ff0 con 0x7f2f0c101540 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.500+0000 7f2f0bfff700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0c101540 0x7f2f0c19d6d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2efc000c00 tx=0x7f2efc00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.500+0000 7f2f097fa700 1 -- 172.21.15.49:0/3545654730 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc027070 con 0x7f2f0c101540 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.500+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0c198280 con 0x7f2f0c101540 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.500+0000 7f2f097fa700 1 -- 172.21.15.49:0/3545654730 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2efc00be70 con 0x7f2f0c101540 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.500+0000 7f2f097fa700 1 -- 172.21.15.49:0/3545654730 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00f460 con 0x7f2f0c101540 2024-03-26T18:00:49.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.500+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0c06aab0 con 0x7f2f0c101540 2024-03-26T18:00:49.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.501+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f0c06b120 con 0x7f2f0c101540 2024-03-26T18:00:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.502+0000 7f2f097fa700 1 -- 172.21.15.49:0/3545654730 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f2efc02a070 con 0x7f2f0c101540 2024-03-26T18:00:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.502+0000 7f2f097fa700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ef40a3040 0x7f2ef40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.503+0000 7f2f0b7fe700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ef40a3040 0x7f2ef40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.503+0000 7f2f097fa700 1 -- 172.21.15.49:0/3545654730 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f2efc0d1120 con 0x7f2f0c101540 2024-03-26T18:00:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.503+0000 7f2f097fa700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] conn(0x7f2ef40ae120 0x7f2ef40b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.503+0000 7f2f097fa700 1 -- 172.21.15.49:0/3545654730 --> [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2efc024ec0 con 0x7f2ef40ae120 2024-03-26T18:00:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.503+0000 7f2f097fa700 1 -- 172.21.15.49:0/3545654730 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f2efc0d1570 con 0x7f2f0c101540 2024-03-26T18:00:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.503+0000 7f2f10963700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] conn(0x7f2ef40ae120 0x7f2ef40b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.503+0000 7f2f0b7fe700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ef40a3040 0x7f2ef40a5500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c100eb0 tx=0x7f2f00010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:49.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.503+0000 7f2f10963700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] conn(0x7f2ef40ae120 0x7f2ef40b05c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:49.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.505+0000 7f2f097fa700 1 -- 172.21.15.49:0/3545654730 <== osd.10 v2:172.21.15.115:6824/2513437602 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2efc024ec0 con 0x7f2ef40ae120 2024-03-26T18:00:49.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.521+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 --> [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f0c06b520 con 0x7f2ef40ae120 2024-03-26T18:00:49.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f097fa700 1 -- 172.21.15.49:0/3545654730 <== osd.10 v2:172.21.15.115:6824/2513437602 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2f0c06b520 con 0x7f2ef40ae120 2024-03-26T18:00:49.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 >> [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] conn(0x7f2ef40ae120 msgr2=0x7f2ef40b05c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:49.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f123c6700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] conn(0x7f2ef40ae120 0x7f2ef40b05c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:49.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ef40a3040 msgr2=0x7f2ef40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:49.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f123c6700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ef40a3040 0x7f2ef40a5500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c100eb0 tx=0x7f2f00010040 comp rx=0 tx=0).stop 2024-03-26T18:00:49.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0c101540 msgr2=0x7f2f0c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:49.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f123c6700 1 --2- 172.21.15.49:0/3545654730 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f0c101540 0x7f2f0c19d6d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2efc000c00 tx=0x7f2efc00bab0 comp rx=0 tx=0).stop 2024-03-26T18:00:49.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f10963700 1 -- 172.21.15.49:0/3545654730 reap_dead start 2024-03-26T18:00:49.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 shutdown_connections 2024-03-26T18:00:49.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 >> 172.21.15.49:0/3545654730 conn(0x7f2f0c0772d0 msgr2=0x7f2f0c107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:49.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 shutdown_connections 2024-03-26T18:00:49.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:49.522+0000 7f2f123c6700 1 -- 172.21.15.49:0/3545654730 wait complete. 2024-03-26T18:00:50.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.235+0000 7fb2a0200700 1 -- 172.21.15.49:0/1286897284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2981009a0 msgr2=0x7fb298100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:50.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.235+0000 7fb2a0200700 1 --2- 172.21.15.49:0/1286897284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2981009a0 0x7fb298100e20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb288009990 tx=0x7fb288009ca0 comp rx=0 tx=0).stop 2024-03-26T18:00:50.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.235+0000 7fb2a0200700 1 -- 172.21.15.49:0/1286897284 shutdown_connections 2024-03-26T18:00:50.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.235+0000 7fb2a0200700 1 --2- 172.21.15.49:0/1286897284 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb298101360 0x7fb29810f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:50.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.235+0000 7fb2a0200700 1 --2- 172.21.15.49:0/1286897284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb298108500 0x7fb2981088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:50.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.235+0000 7fb2a0200700 1 --2- 172.21.15.49:0/1286897284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2981009a0 0x7fb298100e20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:50.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.235+0000 7fb2a0200700 1 -- 172.21.15.49:0/1286897284 >> 172.21.15.49:0/1286897284 conn(0x7fb2980778b0 msgr2=0x7fb2980fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:50.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.235+0000 7fb2a0200700 1 -- 172.21.15.49:0/1286897284 shutdown_connections 2024-03-26T18:00:50.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.235+0000 7fb2a0200700 1 -- 172.21.15.49:0/1286897284 wait complete. 2024-03-26T18:00:50.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb2a0200700 1 Processor -- start 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb2a0200700 1 -- start start 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb2a0200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2981009a0 0x7fb29819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb2a0200700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb298101360 0x7fb29819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb2a0200700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb298108500 0x7fb298198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb2a0200700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981126e0 con 0x7fb298108500 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb2a0200700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb298112560 con 0x7fb298101360 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb2a0200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb298112860 con 0x7fb2981009a0 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb29d79b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb298101360 0x7fb29819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb29d79b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb298101360 0x7fb29819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34484/0 (socket says 172.21.15.49:34484) 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb29d79b700 1 -- 172.21.15.49:0/184391527 learned_addr learned my addr 172.21.15.49:0/184391527 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb29df9c700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2981009a0 0x7fb29819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.236+0000 7fb29e79d700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb298108500 0x7fb298198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb29d79b700 1 -- 172.21.15.49:0/184391527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2981009a0 msgr2=0x7fb29819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:50.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb29d79b700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2981009a0 0x7fb29819e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb29d79b700 1 -- 172.21.15.49:0/184391527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb298108500 msgr2=0x7fb298198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb29d79b700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb298108500 0x7fb298198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb29d79b700 1 -- 172.21.15.49:0/184391527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2981990e0 con 0x7fb298101360 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb29e79d700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb298108500 0x7fb298198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb29d79b700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb298101360 0x7fb29819ecb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb288000c00 tx=0x7fb28800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb28effd700 1 -- 172.21.15.49:0/184391527 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288027070 con 0x7fb298101360 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb28effd700 1 -- 172.21.15.49:0/184391527 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb288003d70 con 0x7fb298101360 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb28effd700 1 -- 172.21.15.49:0/184391527 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28800f460 con 0x7fb298101360 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb298199370 con 0x7fb298101360 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.237+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2981a5640 con 0x7fb298101360 2024-03-26T18:00:50.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.238+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb298062ac0 con 0x7fb298101360 2024-03-26T18:00:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.239+0000 7fb28effd700 1 -- 172.21.15.49:0/184391527 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fb28802a070 con 0x7fb298101360 2024-03-26T18:00:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.240+0000 7fb28effd700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb2840a3040 0x7fb2840a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.240+0000 7fb29df9c700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb2840a3040 0x7fb2840a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.240+0000 7fb28effd700 1 -- 172.21.15.49:0/184391527 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fb2880d0ae0 con 0x7fb298101360 2024-03-26T18:00:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.240+0000 7fb28effd700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] conn(0x7fb2840ae120 0x7fb2840b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.240+0000 7fb28effd700 1 -- 172.21.15.49:0/184391527 --> [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2880043d0 con 0x7fb2840ae120 2024-03-26T18:00:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.240+0000 7fb28effd700 1 -- 172.21.15.49:0/184391527 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb2880d0e90 con 0x7fb298101360 2024-03-26T18:00:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.240+0000 7fb29e79d700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] conn(0x7fb2840ae120 0x7fb2840b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.240+0000 7fb29df9c700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb2840a3040 0x7fb2840a5500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb298107520 tx=0x7fb294009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.241+0000 7fb29e79d700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] conn(0x7fb2840ae120 0x7fb2840b05c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:50.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.242+0000 7fb28effd700 1 -- 172.21.15.49:0/184391527 <== osd.2 v2:172.21.15.171:6800/2543547141 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2880043d0 con 0x7fb2840ae120 2024-03-26T18:00:50.246 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:50.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.266+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 --> [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb298062f40 con 0x7fb2840ae120 2024-03-26T18:00:50.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.267+0000 7fb28effd700 1 -- 172.21.15.49:0/184391527 <== osd.2 v2:172.21.15.171:6800/2543547141 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb298062f40 con 0x7fb2840ae120 2024-03-26T18:00:50.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:50 smithi049 ceph-mon[32331]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-03-26T18:00:50.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:50 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:50 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:50 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:50 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.267+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 >> [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] conn(0x7fb2840ae120 msgr2=0x7fb2840b05c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:50.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.267+0000 7fb2a0200700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] conn(0x7fb2840ae120 0x7fb2840b05c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:50.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.268+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb2840a3040 msgr2=0x7fb2840a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:50.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.268+0000 7fb2a0200700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb2840a3040 0x7fb2840a5500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb298107520 tx=0x7fb294009260 comp rx=0 tx=0).stop 2024-03-26T18:00:50.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.268+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb298101360 msgr2=0x7fb29819ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:50.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.268+0000 7fb2a0200700 1 --2- 172.21.15.49:0/184391527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb298101360 0x7fb29819ecb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb288000c00 tx=0x7fb28800bab0 comp rx=0 tx=0).stop 2024-03-26T18:00:50.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.268+0000 7fb29e79d700 1 -- 172.21.15.49:0/184391527 reap_dead start 2024-03-26T18:00:50.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.268+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 shutdown_connections 2024-03-26T18:00:50.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.268+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 >> 172.21.15.49:0/184391527 conn(0x7fb2980778b0 msgr2=0x7fb2980ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:50.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.268+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 shutdown_connections 2024-03-26T18:00:50.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:50.268+0000 7fb2a0200700 1 -- 172.21.15.49:0/184391527 wait complete. 2024-03-26T18:00:50.487 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:50 smithi115 ceph-mon[36174]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-03-26T18:00:50.522 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:50 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.522 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:50 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.522 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:50 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.522 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:50 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:50 smithi171 ceph-mon[36033]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-03-26T18:00:50.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:50 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:50 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:50 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:50.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:50 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:00:51.308 INFO:teuthology.orchestra.run.smithi049.stdout:313532612638 2024-03-26T18:00:51.308 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-26T18:00:52.428 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:52 smithi049 ceph-mon[32331]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:52.487 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:52 smithi115 ceph-mon[36174]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:52.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:52 smithi171 ceph-mon[36033]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:53.057 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:53.058 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:53.060 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:54.487 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:54 smithi115 ceph-mon[36174]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:54.502 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:54 smithi049 ceph-mon[32331]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:54.598 INFO:teuthology.orchestra.run.smithi049.stdout:77309411400 2024-03-26T18:00:54.598 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-26T18:00:54.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:54 smithi171 ceph-mon[36033]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:56.487 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:56 smithi115 ceph-mon[36174]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:56.502 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:56 smithi049 ceph-mon[32331]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:56.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:56 smithi171 ceph-mon[36033]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:57.525 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:00:58.350 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:58 smithi049 ceph-mon[32331]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:58.487 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:58 smithi115 ceph-mon[36174]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:58.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:58 smithi171 ceph-mon[36033]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:00:59.249 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:00:59 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1835403948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T18:00:59.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:00:59 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1835403948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T18:00:59.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:00:59 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1835403948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T18:01:00.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.214+0000 7f39e4c11700 1 -- 172.21.15.49:0/391554557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e01083e0 msgr2=0x7f39e01087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.214+0000 7f39e4c11700 1 --2- 172.21.15.49:0/391554557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e01083e0 0x7f39e01087c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009920 tx=0x7f39c8009c30 comp rx=0 tx=0).stop 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.214+0000 7f39e4c11700 1 -- 172.21.15.49:0/391554557 shutdown_connections 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.214+0000 7f39e4c11700 1 --2- 172.21.15.49:0/391554557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39e0102dc0 0x7f39e010f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.214+0000 7f39e4c11700 1 --2- 172.21.15.49:0/391554557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e01083e0 0x7f39e01087c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.214+0000 7f39e4c11700 1 --2- 172.21.15.49:0/391554557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0102400 0x7f39e0102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.214+0000 7f39e4c11700 1 -- 172.21.15.49:0/391554557 >> 172.21.15.49:0/391554557 conn(0x7f39e00794e0 msgr2=0x7f39e00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.214+0000 7f39e4c11700 1 -- 172.21.15.49:0/391554557 shutdown_connections 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.214+0000 7f39e4c11700 1 -- 172.21.15.49:0/391554557 wait complete. 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39e4c11700 1 Processor -- start 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39e4c11700 1 -- start start 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39e4c11700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0102400 0x7f39e019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39e4c11700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e0102dc0 0x7f39e019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39e4c11700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39e01083e0 0x7f39e0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39e4c11700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0112660 con 0x7f39e0102400 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39e4c11700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01124e0 con 0x7f39e0102dc0 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39e4c11700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01127e0 con 0x7f39e01083e0 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39de59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0102400 0x7f39e019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39de59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0102400 0x7f39e019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36432/0 (socket says 172.21.15.49:36432) 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.215+0000 7f39de59c700 1 -- 172.21.15.49:0/3528026336 learned_addr learned my addr 172.21.15.49:0/3528026336 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39ddd9b700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e0102dc0 0x7f39e019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39ded9d700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39e01083e0 0x7f39e0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39de59c700 1 -- 172.21.15.49:0/3528026336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39e01083e0 msgr2=0x7f39e0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39de59c700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39e01083e0 0x7f39e0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39de59c700 1 -- 172.21.15.49:0/3528026336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e0102dc0 msgr2=0x7f39e019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39de59c700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e0102dc0 0x7f39e019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39de59c700 1 -- 172.21.15.49:0/3528026336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e0199180 con 0x7f39e0102400 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39de59c700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0102400 0x7f39e019e860 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005e90 tx=0x7f39c800fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39d77fe700 1 -- 172.21.15.49:0/3528026336 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80050c0 con 0x7f39e0102400 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39d77fe700 1 -- 172.21.15.49:0/3528026336 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c800b560 con 0x7f39e0102400 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39e4c11700 1 -- 172.21.15.49:0/3528026336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e0199410 con 0x7f39e0102400 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39d77fe700 1 -- 172.21.15.49:0/3528026336 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8003af0 con 0x7f39e0102400 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.216+0000 7f39e4c11700 1 -- 172.21.15.49:0/3528026336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e01a5730 con 0x7f39e0102400 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.217+0000 7f39e4c11700 1 -- 172.21.15.49:0/3528026336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e0069070 con 0x7f39e0102400 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.218+0000 7f39d77fe700 1 -- 172.21.15.49:0/3528026336 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f39c800baa0 con 0x7f39e0102400 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.219+0000 7f39d77fe700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39cc0a30c0 0x7f39cc0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.219+0000 7f39ddd9b700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39cc0a30c0 0x7f39cc0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.219+0000 7f39d77fe700 1 -- 172.21.15.49:0/3528026336 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f39c80d09d0 con 0x7f39e0102400 2024-03-26T18:01:00.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.220+0000 7f39ddd9b700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39cc0a30c0 0x7f39cc0a5580 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009750 tx=0x7f39d0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.223+0000 7f39d77fe700 1 -- 172.21.15.49:0/3528026336 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f39c8013030 con 0x7f39e0102400 2024-03-26T18:01:00.403 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:00 smithi049 ceph-mon[32331]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:01:00.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.433+0000 7f39e4c11700 1 -- 172.21.15.49:0/3528026336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f39e0199f50 con 0x7f39e0102400 2024-03-26T18:01:00.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.434+0000 7f39d77fe700 1 -- 172.21.15.49:0/3528026336 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f39c800e3e0 con 0x7f39e0102400 2024-03-26T18:01:00.415 INFO:teuthology.orchestra.run.smithi049.stdout:201863462965 2024-03-26T18:01:00.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.435+0000 7f03d3a2e700 1 -- 172.21.15.49:0/2758474459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03cc103040 msgr2=0x7f03cc106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.435+0000 7f03d3a2e700 1 --2- 172.21.15.49:0/2758474459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03cc103040 0x7f03cc106fe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009990 tx=0x7f03bc009ca0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.435+0000 7f03d3a2e700 1 -- 172.21.15.49:0/2758474459 shutdown_connections 2024-03-26T18:01:00.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.435+0000 7f03d3a2e700 1 --2- 172.21.15.49:0/2758474459 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc1076c0 0x7f03cc10f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.435+0000 7f03d3a2e700 1 --2- 172.21.15.49:0/2758474459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc102690 0x7f03cc102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.435+0000 7f03d3a2e700 1 --2- 172.21.15.49:0/2758474459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03cc103040 0x7f03cc106fe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.435+0000 7f03d3a2e700 1 -- 172.21.15.49:0/2758474459 >> 172.21.15.49:0/2758474459 conn(0x7f03cc079440 msgr2=0x7f03cc079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.435+0000 7f03d3a2e700 1 -- 172.21.15.49:0/2758474459 shutdown_connections 2024-03-26T18:01:00.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.435+0000 7f03d3a2e700 1 -- 172.21.15.49:0/2758474459 wait complete. 2024-03-26T18:01:00.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d3a2e700 1 Processor -- start 2024-03-26T18:01:00.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d3a2e700 1 -- start start 2024-03-26T18:01:00.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d3a2e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03cc102690 0x7f03cc19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d3a2e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc103040 0x7f03cc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d3a2e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc1076c0 0x7f03cc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d3a2e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1126b0 con 0x7f03cc1076c0 2024-03-26T18:01:00.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d3a2e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc112530 con 0x7f03cc102690 2024-03-26T18:01:00.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d3a2e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc112830 con 0x7f03cc103040 2024-03-26T18:01:00.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d17ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03cc102690 0x7f03cc19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d17ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03cc102690 0x7f03cc19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40632/0 (socket says 172.21.15.49:40632) 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d17ca700 1 -- 172.21.15.49:0/198962233 learned_addr learned my addr 172.21.15.49:0/198962233 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d0fc9700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc103040 0x7f03cc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.436+0000 7f03d1fcb700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc1076c0 0x7f03cc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03d17ca700 1 -- 172.21.15.49:0/198962233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc103040 msgr2=0x7f03cc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03d17ca700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc103040 0x7f03cc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03d17ca700 1 -- 172.21.15.49:0/198962233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc1076c0 msgr2=0x7f03cc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03d17ca700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc1076c0 0x7f03cc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03d17ca700 1 -- 172.21.15.49:0/198962233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc199110 con 0x7f03cc102690 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03d0fc9700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc103040 0x7f03cc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 -- 172.21.15.49:0/3528026336 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39cc0a30c0 msgr2=0x7f39cc0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39cc0a30c0 0x7f39cc0a5580 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009750 tx=0x7f39d0009310 comp rx=0 tx=0).stop 2024-03-26T18:01:00.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 -- 172.21.15.49:0/3528026336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0102400 msgr2=0x7f39e019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0102400 0x7f39e019e860 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005e90 tx=0x7f39c800fa90 comp rx=0 tx=0).stop 2024-03-26T18:01:00.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03d17ca700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03cc102690 0x7f03cc19e7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f03c800e9a0 tx=0x7f03c800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 -- 172.21.15.49:0/3528026336 shutdown_connections 2024-03-26T18:01:00.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f39cc0a30c0 0x7f39cc0a5580 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f39e01083e0 0x7f39e0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39e0102400 0x7f39e019e860 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 --2- 172.21.15.49:0/3528026336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39e0102dc0 0x7f39e019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 -- 172.21.15.49:0/3528026336 >> 172.21.15.49:0/3528026336 conn(0x7f39e00794e0 msgr2=0x7f39e0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03c27fc700 1 -- 172.21.15.49:0/198962233 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c800ca10 con 0x7f03cc102690 2024-03-26T18:01:00.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03c27fc700 1 -- 172.21.15.49:0/198962233 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c800cbb0 con 0x7f03cc102690 2024-03-26T18:01:00.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03c27fc700 1 -- 172.21.15.49:0/198962233 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8018680 con 0x7f03cc102690 2024-03-26T18:01:00.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03d3a2e700 1 -- 172.21.15.49:0/198962233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc199400 con 0x7f03cc102690 2024-03-26T18:01:00.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 -- 172.21.15.49:0/3528026336 shutdown_connections 2024-03-26T18:01:00.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f39e4c11700 1 -- 172.21.15.49:0/3528026336 wait complete. 2024-03-26T18:01:00.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.437+0000 7f03d3a2e700 1 -- 172.21.15.49:0/198962233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc0745e0 con 0x7f03cc102690 2024-03-26T18:01:00.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.438+0000 7f03d3a2e700 1 -- 172.21.15.49:0/198962233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03cc069070 con 0x7f03cc102690 2024-03-26T18:01:00.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.440+0000 7f03c27fc700 1 -- 172.21.15.49:0/198962233 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f03c8010430 con 0x7f03cc102690 2024-03-26T18:01:00.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.441+0000 7f03c27fc700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03b80a30c0 0x7f03b80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.441+0000 7f03d0fc9700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03b80a30c0 0x7f03b80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.441+0000 7f03c27fc700 1 -- 172.21.15.49:0/198962233 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f03c80cfaa0 con 0x7f03cc102690 2024-03-26T18:01:00.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.442+0000 7f03d0fc9700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03b80a30c0 0x7f03b80a5580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f03cc102010 tx=0x7f03bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.445+0000 7f03c27fc700 1 -- 172.21.15.49:0/198962233 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f03c809c3d0 con 0x7f03cc102690 2024-03-26T18:01:00.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.014+0000 7fec5ef21700 1 -- 172.21.15.49:0/767846680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec58101c50 msgr2=0x7fec5810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.014+0000 7fec5ef21700 1 --2- 172.21.15.49:0/767846680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec58101c50 0x7fec5810e130 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fec5000b0a0 tx=0x7fec5000b3b0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.014+0000 7fec5ef21700 1 -- 172.21.15.49:0/767846680 shutdown_connections 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.014+0000 7fec5ef21700 1 --2- 172.21.15.49:0/767846680 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec58101c50 0x7fec5810e130 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.014+0000 7fec5ef21700 1 --2- 172.21.15.49:0/767846680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec581072f0 0x7fec581076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.014+0000 7fec5ef21700 1 --2- 172.21.15.49:0/767846680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec581012b0 0x7fec58101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.014+0000 7fec5ef21700 1 -- 172.21.15.49:0/767846680 >> 172.21.15.49:0/767846680 conn(0x7fec580781e0 msgr2=0x7fec580785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.014+0000 7fec5ef21700 1 -- 172.21.15.49:0/767846680 shutdown_connections 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.014+0000 7fec5ef21700 1 -- 172.21.15.49:0/767846680 wait complete. 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.015+0000 7fec5ef21700 1 Processor -- start 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.015+0000 7fec5ef21700 1 -- start start 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ef21700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec581012b0 0x7fec58199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ef21700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec58101c50 0x7fec58199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ef21700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec581072f0 0x7fec5819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ef21700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec58111540 con 0x7fec581072f0 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ef21700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fec581113c0 con 0x7fec581012b0 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ef21700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fec581116c0 con 0x7fec58101c50 2024-03-26T18:01:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ccbd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec581012b0 0x7fec58199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5d4be700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec581072f0 0x7fec5819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec57fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec58101c50 0x7fec58199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ccbd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec581012b0 0x7fec58199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40604/0 (socket says 172.21.15.49:40604) 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ccbd700 1 -- 172.21.15.49:0/1835403948 learned_addr learned my addr 172.21.15.49:0/1835403948 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5d4be700 1 -- 172.21.15.49:0/1835403948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec58101c50 msgr2=0x7fec58199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5d4be700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec58101c50 0x7fec58199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5d4be700 1 -- 172.21.15.49:0/1835403948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec581012b0 msgr2=0x7fec58199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5d4be700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec581012b0 0x7fec58199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5d4be700 1 -- 172.21.15.49:0/1835403948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec581a4e40 con 0x7fec581072f0 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ccbd700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec581012b0 0x7fec58199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5d4be700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec581072f0 0x7fec5819a4c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fec5000b0a0 tx=0x7fec50004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.531 INFO:teuthology.orchestra.run.smithi049.stdout:111669149764 2024-03-26T18:01:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec55ffb700 1 -- 172.21.15.49:0/1835403948 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50021070 con 0x7fec581072f0 2024-03-26T18:01:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec55ffb700 1 -- 172.21.15.49:0/1835403948 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec50007b80 con 0x7fec581072f0 2024-03-26T18:01:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec5ef21700 1 -- 172.21.15.49:0/1835403948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec581a50d0 con 0x7fec581072f0 2024-03-26T18:01:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.016+0000 7fec55ffb700 1 -- 172.21.15.49:0/1835403948 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec500052a0 con 0x7fec581072f0 2024-03-26T18:01:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.017+0000 7fec5ef21700 1 -- 172.21.15.49:0/1835403948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec581a5580 con 0x7fec581072f0 2024-03-26T18:01:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.017+0000 7fec5ef21700 1 -- 172.21.15.49:0/1835403948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58069070 con 0x7fec581072f0 2024-03-26T18:01:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.019+0000 7fec55ffb700 1 -- 172.21.15.49:0/1835403948 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fec50024070 con 0x7fec581072f0 2024-03-26T18:01:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.019+0000 7fec55ffb700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec480a3040 0x7fec480a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.019+0000 7fec5ccbd700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec480a3040 0x7fec480a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.019+0000 7fec55ffb700 1 -- 172.21.15.49:0/1835403948 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fec5000f020 con 0x7fec581072f0 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.020+0000 7fec5ccbd700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec480a3040 0x7fec480a5500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fec4c011550 tx=0x7fec4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.023+0000 7fec55ffb700 1 -- 172.21.15.49:0/1835403948 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fec5002b020 con 0x7fec581072f0 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.234+0000 7fec5ef21700 1 -- 172.21.15.49:0/1835403948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fec5802dc00 con 0x7fec581072f0 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.234+0000 7fec55ffb700 1 -- 172.21.15.49:0/1835403948 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fec5009dc20 con 0x7fec581072f0 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 -- 172.21.15.49:0/1835403948 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec480a3040 msgr2=0x7fec480a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec480a3040 0x7fec480a5500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fec4c011550 tx=0x7fec4c009260 comp rx=0 tx=0).stop 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 -- 172.21.15.49:0/1835403948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec581072f0 msgr2=0x7fec5819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec581072f0 0x7fec5819a4c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fec5000b0a0 tx=0x7fec50004d00 comp rx=0 tx=0).stop 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 -- 172.21.15.49:0/1835403948 shutdown_connections 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fec480a3040 0x7fec480a5500 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec581072f0 0x7fec5819a4c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec581012b0 0x7fec58199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 --2- 172.21.15.49:0/1835403948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec58101c50 0x7fec58199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 -- 172.21.15.49:0/1835403948 >> 172.21.15.49:0/1835403948 conn(0x7fec580781e0 msgr2=0x7fec580ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.236+0000 7fec5ef21700 1 -- 172.21.15.49:0/1835403948 shutdown_connections 2024-03-26T18:01:00.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:00:59.237+0000 7fec5ef21700 1 -- 172.21.15.49:0/1835403948 wait complete. 2024-03-26T18:01:00.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:00 smithi171 ceph-mon[36033]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:01:00.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.654+0000 7f03d3a2e700 1 -- 172.21.15.49:0/198962233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f03cc02dc00 con 0x7f03cc102690 2024-03-26T18:01:00.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.654+0000 7f03c27fc700 1 -- 172.21.15.49:0/198962233 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f03c809bcd0 con 0x7f03cc102690 2024-03-26T18:01:00.636 INFO:teuthology.orchestra.run.smithi049.stdout:146028888127 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 -- 172.21.15.49:0/198962233 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03b80a30c0 msgr2=0x7f03b80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03b80a30c0 0x7f03b80a5580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f03cc102010 tx=0x7f03bc005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 -- 172.21.15.49:0/198962233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03cc102690 msgr2=0x7f03cc19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03cc102690 0x7f03cc19e7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f03c800e9a0 tx=0x7f03c800ecb0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 -- 172.21.15.49:0/198962233 shutdown_connections 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03b80a30c0 0x7f03b80a5580 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03cc1076c0 0x7f03cc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03cc102690 0x7f03cc19e7f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 --2- 172.21.15.49:0/198962233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03cc103040 0x7f03cc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 -- 172.21.15.49:0/198962233 >> 172.21.15.49:0/198962233 conn(0x7f03cc079440 msgr2=0x7f03cc108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 -- 172.21.15.49:0/198962233 shutdown_connections 2024-03-26T18:01:00.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:00.657+0000 7f03d3a2e700 1 -- 172.21.15.49:0/198962233 wait complete. 2024-03-26T18:01:00.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:00 smithi115 ceph-mon[36174]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:01:01.225 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149758 got 111669149764 for osd.3 2024-03-26T18:01:01.225 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:01.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:01 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3528026336' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T18:01:01.487 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:01 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/198962233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T18:01:01.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:01 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3528026336' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T18:01:01.502 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:01 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/198962233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T18:01:01.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:01 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3528026336' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T18:01:01.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:01 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/198962233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T18:01:01.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.747+0000 7f55569a8700 1 -- 172.21.15.49:0/3549167739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5550106590 msgr2=0x7f555010e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:01.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.747+0000 7f55569a8700 1 --2- 172.21.15.49:0/3549167739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5550106590 0x7f555010e160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f554800b0a0 tx=0x7f554800b3b0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.747+0000 7f55569a8700 1 -- 172.21.15.49:0/3549167739 shutdown_connections 2024-03-26T18:01:01.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.747+0000 7f55569a8700 1 --2- 172.21.15.49:0/3549167739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5550106590 0x7f555010e160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.747+0000 7f55569a8700 1 --2- 172.21.15.49:0/3549167739 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5550101540 0x7f5550101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.747+0000 7f55569a8700 1 --2- 172.21.15.49:0/3549167739 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5550101ef0 0x7f5550105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.747+0000 7f55569a8700 1 -- 172.21.15.49:0/3549167739 >> 172.21.15.49:0/3549167739 conn(0x7f5550077300 msgr2=0x7f5550077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:01.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.747+0000 7f55569a8700 1 -- 172.21.15.49:0/3549167739 shutdown_connections 2024-03-26T18:01:01.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.748+0000 7f55569a8700 1 -- 172.21.15.49:0/3549167739 wait complete. 2024-03-26T18:01:01.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.748+0000 7f55569a8700 1 Processor -- start 2024-03-26T18:01:01.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.748+0000 7f55569a8700 1 -- start start 2024-03-26T18:01:01.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.748+0000 7f55569a8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5550101540 0x7f555019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:01.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f55569a8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5550101ef0 0x7f555019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:01.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f55569a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5550106590 0x7f5550198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:01.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f55569a8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f55501115a0 con 0x7f5550101540 2024-03-26T18:01:01.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f55569a8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5550111420 con 0x7f5550106590 2024-03-26T18:01:01.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f55569a8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5550111720 con 0x7f5550101ef0 2024-03-26T18:01:01.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f5554f45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5550106590 0x7f5550198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f554ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5550101540 0x7f555019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f554f7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5550101ef0 0x7f555019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f5554f45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5550106590 0x7f5550198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:40668/0 (socket says 172.21.15.49:40668) 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f5554f45700 1 -- 172.21.15.49:0/2192487636 learned_addr learned my addr 172.21.15.49:0/2192487636 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f554f7fe700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5550101ef0 0x7f555019ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:40624/0 (socket says 172.21.15.49:40624) 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f5554f45700 1 -- 172.21.15.49:0/2192487636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5550101ef0 msgr2=0x7f555019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f5554f45700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5550101ef0 0x7f555019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f5554f45700 1 -- 172.21.15.49:0/2192487636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5550101540 msgr2=0x7f555019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f5554f45700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5550101540 0x7f555019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f5554f45700 1 -- 172.21.15.49:0/2192487636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5550199080 con 0x7f5550106590 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f5554f45700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5550106590 0x7f5550198920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5548006070 tx=0x7f5548004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:01.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f554d7fa700 1 -- 172.21.15.49:0/2192487636 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548021070 con 0x7f5550106590 2024-03-26T18:01:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f554d7fa700 1 -- 172.21.15.49:0/2192487636 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5548003c00 con 0x7f5550106590 2024-03-26T18:01:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f55569a8700 1 -- 172.21.15.49:0/2192487636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5550199310 con 0x7f5550106590 2024-03-26T18:01:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.749+0000 7f554d7fa700 1 -- 172.21.15.49:0/2192487636 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55480047a0 con 0x7f5550106590 2024-03-26T18:01:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.750+0000 7f55569a8700 1 -- 172.21.15.49:0/2192487636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55501a55e0 con 0x7f5550106590 2024-03-26T18:01:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.750+0000 7f55569a8700 1 -- 172.21.15.49:0/2192487636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5550069070 con 0x7f5550106590 2024-03-26T18:01:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.752+0000 7f554d7fa700 1 -- 172.21.15.49:0/2192487636 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f5548007980 con 0x7f5550106590 2024-03-26T18:01:01.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.753+0000 7f554d7fa700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f553c0a3040 0x7f553c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:01.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.753+0000 7f554ffff700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f553c0a3040 0x7f553c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:01.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.753+0000 7f554d7fa700 1 -- 172.21.15.49:0/2192487636 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f554800f020 con 0x7f5550106590 2024-03-26T18:01:01.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.754+0000 7f554ffff700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f553c0a3040 0x7f553c0a5500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5550077b10 tx=0x7f5540009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:01.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.757+0000 7f554d7fa700 1 -- 172.21.15.49:0/2192487636 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f554802a020 con 0x7f5550106590 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.653+0000 7f7214c24700 1 -- 172.21.15.49:0/558039662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7210102430 msgr2=0x7f72101028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.653+0000 7f7214c24700 1 --2- 172.21.15.49:0/558039662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7210102430 0x7f72101028b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7204009990 tx=0x7f7204009ca0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.653+0000 7f7214c24700 1 -- 172.21.15.49:0/558039662 shutdown_connections 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.653+0000 7f7214c24700 1 --2- 172.21.15.49:0/558039662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7210102df0 0x7f721010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.653+0000 7f7214c24700 1 --2- 172.21.15.49:0/558039662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7210108510 0x7f72101088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.653+0000 7f7214c24700 1 --2- 172.21.15.49:0/558039662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7210102430 0x7f72101028b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.653+0000 7f7214c24700 1 -- 172.21.15.49:0/558039662 >> 172.21.15.49:0/558039662 conn(0x7f7210079470 msgr2=0x7f7210079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.653+0000 7f7214c24700 1 -- 172.21.15.49:0/558039662 shutdown_connections 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.653+0000 7f7214c24700 1 -- 172.21.15.49:0/558039662 wait complete. 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.654+0000 7f7214c24700 1 Processor -- start 2024-03-26T18:01:01.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.654+0000 7f7214c24700 1 -- start start 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f7214c24700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7210102430 0x7f721019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f7214c24700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7210102df0 0x7f721019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f7214c24700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7210108510 0x7f7210198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f7214c24700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7210112690 con 0x7f7210102430 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f7214c24700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7210112510 con 0x7f7210108510 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f7214c24700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7210112810 con 0x7f7210102df0 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7210102430 0x7f721019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7210102430 0x7f721019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36456/0 (socket says 172.21.15.49:36456) 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720e59c700 1 -- 172.21.15.49:0/101069653 learned_addr learned my addr 172.21.15.49:0/101069653 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720dd9b700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7210102df0 0x7f721019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720ed9d700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7210108510 0x7f7210198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720e59c700 1 -- 172.21.15.49:0/101069653 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7210102df0 msgr2=0x7f721019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720e59c700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7210102df0 0x7f721019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720e59c700 1 -- 172.21.15.49:0/101069653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7210108510 msgr2=0x7f7210198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720e59c700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7210108510 0x7f7210198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720e59c700 1 -- 172.21.15.49:0/101069653 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72101990b0 con 0x7f7210102430 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.655+0000 7f720ed9d700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7210108510 0x7f7210198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.656+0000 7f720e59c700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7210102430 0x7f721019e790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f720000e9a0 tx=0x7f720000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.656+0000 7f71ff7fe700 1 -- 172.21.15.49:0/101069653 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720000cbd0 con 0x7f7210102430 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.656+0000 7f7214c24700 1 -- 172.21.15.49:0/101069653 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72101993a0 con 0x7f7210102430 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.656+0000 7f71ff7fe700 1 -- 172.21.15.49:0/101069653 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f720000cd70 con 0x7f7210102430 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.656+0000 7f71ff7fe700 1 -- 172.21.15.49:0/101069653 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200018750 con 0x7f7210102430 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.656+0000 7f7214c24700 1 -- 172.21.15.49:0/101069653 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72100745e0 con 0x7f7210102430 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.657+0000 7f7214c24700 1 -- 172.21.15.49:0/101069653 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7210069070 con 0x7f7210102430 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.658+0000 7f71ff7fe700 1 -- 172.21.15.49:0/101069653 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f7200010430 con 0x7f7210102430 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.659+0000 7f71ff7fe700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f71f80a3040 0x7f71f80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.659+0000 7f720dd9b700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f71f80a3040 0x7f71f80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.659+0000 7f71ff7fe700 1 -- 172.21.15.49:0/101069653 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f72000cfb10 con 0x7f7210102430 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.659+0000 7f720dd9b700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f71f80a3040 0x7f71f80a5500 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f720400b2e0 tx=0x7f7204005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:01.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.664+0000 7f71ff7fe700 1 -- 172.21.15.49:0/101069653 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f720009c4a0 con 0x7f7210102430 2024-03-26T18:01:01.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.874+0000 7f7214c24700 1 -- 172.21.15.49:0/101069653 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f721002dc00 con 0x7f7210102430 2024-03-26T18:01:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.874+0000 7f71ff7fe700 1 -- 172.21.15.49:0/101069653 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f720009bda0 con 0x7f7210102430 2024-03-26T18:01:01.856 INFO:teuthology.orchestra.run.smithi049.stdout:292057776166 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.876+0000 7f7214c24700 1 -- 172.21.15.49:0/101069653 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f71f80a3040 msgr2=0x7f71f80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.876+0000 7f7214c24700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f71f80a3040 0x7f71f80a5500 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f720400b2e0 tx=0x7f7204005960 comp rx=0 tx=0).stop 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.876+0000 7f7214c24700 1 -- 172.21.15.49:0/101069653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7210102430 msgr2=0x7f721019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.876+0000 7f7214c24700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7210102430 0x7f721019e790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f720000e9a0 tx=0x7f720000ecb0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.876+0000 7f7214c24700 1 -- 172.21.15.49:0/101069653 shutdown_connections 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.876+0000 7f7214c24700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f71f80a3040 0x7f71f80a5500 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.876+0000 7f7214c24700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7210108510 0x7f7210198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.876+0000 7f7214c24700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7210102430 0x7f721019e790 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.876+0000 7f7214c24700 1 --2- 172.21.15.49:0/101069653 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7210102df0 0x7f721019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.876+0000 7f7214c24700 1 -- 172.21.15.49:0/101069653 >> 172.21.15.49:0/101069653 conn(0x7f7210079470 msgr2=0x7f72101010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.877+0000 7f7214c24700 1 -- 172.21.15.49:0/101069653 shutdown_connections 2024-03-26T18:01:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.877+0000 7f7214c24700 1 -- 172.21.15.49:0/101069653 wait complete. 2024-03-26T18:01:01.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.967+0000 7f55569a8700 1 -- 172.21.15.49:0/2192487636 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f555002dc00 con 0x7f5550106590 2024-03-26T18:01:01.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.968+0000 7f554d7fa700 1 -- 172.21.15.49:0/2192487636 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f55480a3070 con 0x7f5550106590 2024-03-26T18:01:01.950 INFO:teuthology.orchestra.run.smithi049.stdout:223338299440 2024-03-26T18:01:01.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 -- 172.21.15.49:0/2192487636 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f553c0a3040 msgr2=0x7f553c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:01.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f553c0a3040 0x7f553c0a5500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5550077b10 tx=0x7f5540009260 comp rx=0 tx=0).stop 2024-03-26T18:01:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 -- 172.21.15.49:0/2192487636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5550106590 msgr2=0x7f5550198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5550106590 0x7f5550198920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5548006070 tx=0x7f5548004d00 comp rx=0 tx=0).stop 2024-03-26T18:01:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 -- 172.21.15.49:0/2192487636 shutdown_connections 2024-03-26T18:01:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f553c0a3040 0x7f553c0a5500 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5550106590 0x7f5550198920 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5550101540 0x7f555019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 --2- 172.21.15.49:0/2192487636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5550101ef0 0x7f555019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 -- 172.21.15.49:0/2192487636 >> 172.21.15.49:0/2192487636 conn(0x7f5550077300 msgr2=0x7f555006ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 -- 172.21.15.49:0/2192487636 shutdown_connections 2024-03-26T18:01:01.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:01.970+0000 7f55569a8700 1 -- 172.21.15.49:0/2192487636 wait complete. 2024-03-26T18:01:02.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:02 smithi049 ceph-mon[32331]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:01:02.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:02 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/101069653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T18:01:02.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:02 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2192487636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T18:01:02.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:02 smithi171 ceph-mon[36033]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:01:02.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:02 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/101069653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T18:01:02.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:02 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2192487636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T18:01:02.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:02 smithi115 ceph-mon[36174]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T18:01:02.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:02 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/101069653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T18:01:02.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:02 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2192487636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T18:01:03.493 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:04.127 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299440 for osd.7 2024-03-26T18:01:04.127 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:04.502 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:04 smithi049 ceph-mon[32331]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:04.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:04 smithi171 ceph-mon[36033]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:04.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:04 smithi115 ceph-mon[36174]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:04.968 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776162 got 292057776166 for osd.9 2024-03-26T18:01:04.968 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:05.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.712+0000 7eff2059d700 1 -- 172.21.15.49:0/1409004741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff18074f00 msgr2=0x7eff1810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.712+0000 7eff2059d700 1 --2- 172.21.15.49:0/1409004741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff18074f00 0x7eff1810e870 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7eff08009990 tx=0x7eff08009ca0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.713+0000 7eff2059d700 1 -- 172.21.15.49:0/1409004741 shutdown_connections 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.713+0000 7eff2059d700 1 --2- 172.21.15.49:0/1409004741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff1810edb0 0x7eff181111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.713+0000 7eff2059d700 1 --2- 172.21.15.49:0/1409004741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff180745e0 0x7eff180749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.713+0000 7eff2059d700 1 --2- 172.21.15.49:0/1409004741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff18074f00 0x7eff1810e870 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.713+0000 7eff2059d700 1 -- 172.21.15.49:0/1409004741 >> 172.21.15.49:0/1409004741 conn(0x7eff180794e0 msgr2=0x7eff180798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.713+0000 7eff2059d700 1 -- 172.21.15.49:0/1409004741 shutdown_connections 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.713+0000 7eff2059d700 1 -- 172.21.15.49:0/1409004741 wait complete. 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.713+0000 7eff2059d700 1 Processor -- start 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.713+0000 7eff2059d700 1 -- start start 2024-03-26T18:01:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff2059d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff180745e0 0x7eff1819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff2059d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff18074f00 0x7eff1819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff2059d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1810edb0 0x7eff1819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff2059d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff181148f0 con 0x7eff18074f00 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff2059d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7eff18114770 con 0x7eff1810edb0 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff2059d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7eff18114a70 con 0x7eff180745e0 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1db38700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff18074f00 0x7eff1819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1db38700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff18074f00 0x7eff1819b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36474/0 (socket says 172.21.15.49:36474) 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1db38700 1 -- 172.21.15.49:0/3649396809 learned_addr learned my addr 172.21.15.49:0/3649396809 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1eb3a700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1810edb0 0x7eff1819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1e339700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff180745e0 0x7eff1819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1db38700 1 -- 172.21.15.49:0/3649396809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff180745e0 msgr2=0x7eff1819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1db38700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff180745e0 0x7eff1819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1db38700 1 -- 172.21.15.49:0/3649396809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1810edb0 msgr2=0x7eff1819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1db38700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1810edb0 0x7eff1819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1db38700 1 -- 172.21.15.49:0/3649396809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1819fd30 con 0x7eff18074f00 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff1db38700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff18074f00 0x7eff1819b260 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7eff08000c00 tx=0x7eff0800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff0f7fe700 1 -- 172.21.15.49:0/3649396809 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff08027070 con 0x7eff18074f00 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff2059d700 1 -- 172.21.15.49:0/3649396809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1819ffc0 con 0x7eff18074f00 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff0f7fe700 1 -- 172.21.15.49:0/3649396809 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff08003d70 con 0x7eff18074f00 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff0f7fe700 1 -- 172.21.15.49:0/3649396809 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0800f460 con 0x7eff18074f00 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.714+0000 7eff2059d700 1 -- 172.21.15.49:0/3649396809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff181a78a0 con 0x7eff18074f00 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.715+0000 7eff2059d700 1 -- 172.21.15.49:0/3649396809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff18069070 con 0x7eff18074f00 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.717+0000 7eff0f7fe700 1 -- 172.21.15.49:0/3649396809 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7eff0802a070 con 0x7eff18074f00 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.717+0000 7eff0f7fe700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff040a3040 0x7eff040a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.717+0000 7eff0f7fe700 1 -- 172.21.15.49:0/3649396809 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7eff080d0a20 con 0x7eff18074f00 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.717+0000 7eff1e339700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff040a3040 0x7eff040a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.718+0000 7eff1e339700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff040a3040 0x7eff040a5500 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7eff14009c60 tx=0x7eff14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.721+0000 7eff0f7fe700 1 -- 172.21.15.49:0/3649396809 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7eff08013070 con 0x7eff18074f00 2024-03-26T18:01:05.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.936+0000 7eff2059d700 1 -- 172.21.15.49:0/3649396809 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7eff1802dc00 con 0x7eff18074f00 2024-03-26T18:01:05.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.936+0000 7eff0f7fe700 1 -- 172.21.15.49:0/3649396809 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7eff0809d440 con 0x7eff18074f00 2024-03-26T18:01:05.918 INFO:teuthology.orchestra.run.smithi049.stdout:257698037802 2024-03-26T18:01:05.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 -- 172.21.15.49:0/3649396809 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff040a3040 msgr2=0x7eff040a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff040a3040 0x7eff040a5500 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7eff14009c60 tx=0x7eff14009260 comp rx=0 tx=0).stop 2024-03-26T18:01:05.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 -- 172.21.15.49:0/3649396809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff18074f00 msgr2=0x7eff1819b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff18074f00 0x7eff1819b260 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7eff08000c00 tx=0x7eff0800bab0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 -- 172.21.15.49:0/3649396809 shutdown_connections 2024-03-26T18:01:05.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7eff040a3040 0x7eff040a5500 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1810edb0 0x7eff1819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff180745e0 0x7eff1819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 --2- 172.21.15.49:0/3649396809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff18074f00 0x7eff1819b260 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 -- 172.21.15.49:0/3649396809 >> 172.21.15.49:0/3649396809 conn(0x7eff180794e0 msgr2=0x7eff180ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 -- 172.21.15.49:0/3649396809 shutdown_connections 2024-03-26T18:01:05.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.939+0000 7eff2059d700 1 -- 172.21.15.49:0/3649396809 wait complete. 2024-03-26T18:01:05.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.967+0000 7f1e0a434700 1 -- 172.21.15.49:0/2439436550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e04101ef0 msgr2=0x7f1e04105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.967+0000 7f1e0a434700 1 --2- 172.21.15.49:0/2439436550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e04101ef0 0x7f1e04105eb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1df80099c0 tx=0x7f1df8009cd0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.967+0000 7f1e0a434700 1 -- 172.21.15.49:0/2439436550 shutdown_connections 2024-03-26T18:01:05.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.967+0000 7f1e0a434700 1 --2- 172.21.15.49:0/2439436550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04106590 0x7f1e0410e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.967+0000 7f1e0a434700 1 --2- 172.21.15.49:0/2439436550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e04101540 0x7f1e04101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.967+0000 7f1e0a434700 1 --2- 172.21.15.49:0/2439436550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e04101ef0 0x7f1e04105eb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.967+0000 7f1e0a434700 1 -- 172.21.15.49:0/2439436550 >> 172.21.15.49:0/2439436550 conn(0x7f1e04077300 msgr2=0x7f1e04077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.967+0000 7f1e0a434700 1 -- 172.21.15.49:0/2439436550 shutdown_connections 2024-03-26T18:01:05.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.967+0000 7f1e0a434700 1 -- 172.21.15.49:0/2439436550 wait complete. 2024-03-26T18:01:05.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.968+0000 7f1e0a434700 1 Processor -- start 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.968+0000 7f1e0a434700 1 -- start start 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.968+0000 7f1e0a434700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e04101540 0x7f1e0419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.968+0000 7f1e0a434700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04101ef0 0x7f1e0419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.968+0000 7f1e0a434700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e04106590 0x7f1e041978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.968+0000 7f1e0a434700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041115c0 con 0x7f1e04106590 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.968+0000 7f1e0a434700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04111440 con 0x7f1e04101540 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.968+0000 7f1e0a434700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04111740 con 0x7f1e04101ef0 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e089d1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e04106590 0x7f1e041978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e089d1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e04106590 0x7f1e041978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36488/0 (socket says 172.21.15.49:36488) 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e089d1700 1 -- 172.21.15.49:0/1677264327 learned_addr learned my addr 172.21.15.49:0/1677264327 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e03fff700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e04101540 0x7f1e0419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e037fe700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04101ef0 0x7f1e0419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e03fff700 1 -- 172.21.15.49:0/1677264327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04101ef0 msgr2=0x7f1e0419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e03fff700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04101ef0 0x7f1e0419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e03fff700 1 -- 172.21.15.49:0/1677264327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e04106590 msgr2=0x7f1e041978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e03fff700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e04106590 0x7f1e041978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e03fff700 1 -- 172.21.15.49:0/1677264327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e04198020 con 0x7f1e04101540 2024-03-26T18:01:05.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.969+0000 7f1e03fff700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e04101540 0x7f1e0419d700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1df400e9f0 tx=0x7f1df400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.970+0000 7f1e017fa700 1 -- 172.21.15.49:0/1677264327 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df400c9c0 con 0x7f1e04101540 2024-03-26T18:01:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.970+0000 7f1e017fa700 1 -- 172.21.15.49:0/1677264327 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df400cb60 con 0x7f1e04101540 2024-03-26T18:01:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.970+0000 7f1e017fa700 1 -- 172.21.15.49:0/1677264327 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4018680 con 0x7f1e04101540 2024-03-26T18:01:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.970+0000 7f1e0a434700 1 -- 172.21.15.49:0/1677264327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e04198310 con 0x7f1e04101540 2024-03-26T18:01:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.970+0000 7f1e0a434700 1 -- 172.21.15.49:0/1677264327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0406a8f0 con 0x7f1e04101540 2024-03-26T18:01:05.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.972+0000 7f1e0a434700 1 -- 172.21.15.49:0/1677264327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e04069070 con 0x7f1e04101540 2024-03-26T18:01:05.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.973+0000 7f1e017fa700 1 -- 172.21.15.49:0/1677264327 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f1df4010430 con 0x7f1e04101540 2024-03-26T18:01:05.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.973+0000 7f1e017fa700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1dec0a5420 0x7f1dec0a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.973+0000 7f1e017fa700 1 -- 172.21.15.49:0/1677264327 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f1df40cf950 con 0x7f1e04101540 2024-03-26T18:01:05.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.974+0000 7f1e037fe700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1dec0a5420 0x7f1dec0a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.974+0000 7f1e037fe700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1dec0a5420 0x7f1dec0a78e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000c00 tx=0x7f1df8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:05.979+0000 7f1e017fa700 1 -- 172.21.15.49:0/1677264327 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1df409c280 con 0x7f1e04101540 2024-03-26T18:01:06.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.186+0000 7f1e0a434700 1 -- 172.21.15.49:0/1677264327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1e0406b290 con 0x7f1e04101540 2024-03-26T18:01:06.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.186+0000 7f1e017fa700 1 -- 172.21.15.49:0/1677264327 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1df409bb80 con 0x7f1e04101540 2024-03-26T18:01:06.168 INFO:teuthology.orchestra.run.smithi049.stdout:34359738451 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 -- 172.21.15.49:0/1677264327 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1dec0a5420 msgr2=0x7f1dec0a78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1dec0a5420 0x7f1dec0a78e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000c00 tx=0x7f1df8005fb0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 -- 172.21.15.49:0/1677264327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e04101540 msgr2=0x7f1e0419d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e04101540 0x7f1e0419d700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1df400e9f0 tx=0x7f1df400ed00 comp rx=0 tx=0).stop 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 -- 172.21.15.49:0/1677264327 shutdown_connections 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1dec0a5420 0x7f1dec0a78e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e04106590 0x7f1e041978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e04101540 0x7f1e0419d700 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 --2- 172.21.15.49:0/1677264327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04101ef0 0x7f1e0419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 -- 172.21.15.49:0/1677264327 >> 172.21.15.49:0/1677264327 conn(0x7f1e04077300 msgr2=0x7f1e040ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 -- 172.21.15.49:0/1677264327 shutdown_connections 2024-03-26T18:01:06.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.189+0000 7f1e0a434700 1 -- 172.21.15.49:0/1677264327 wait complete. 2024-03-26T18:01:06.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:06 smithi049 ceph-mon[32331]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-26T18:01:06.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:06 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3649396809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T18:01:06.420 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:06 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1677264327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T18:01:06.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:06 smithi171 ceph-mon[36033]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-26T18:01:06.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:06 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3649396809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T18:01:06.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:06 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1677264327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T18:01:06.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:06 smithi115 ceph-mon[36174]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-26T18:01:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:06 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3649396809' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T18:01:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:06 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1677264327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T18:01:06.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.855+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/3425312454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eec101770 msgr2=0x7f2eec10f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:06.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.855+0000 7f2ef0e3e700 1 --2- 172.21.15.49:0/3425312454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eec101770 0x7f2eec10f300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400b0d0 tx=0x7f2ee400b3e0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.856+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/3425312454 shutdown_connections 2024-03-26T18:01:06.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.856+0000 7f2ef0e3e700 1 --2- 172.21.15.49:0/3425312454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eec101770 0x7f2eec10f300 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.856+0000 7f2ef0e3e700 1 --2- 172.21.15.49:0/3425312454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eec107480 0x7f2eec107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.856+0000 7f2ef0e3e700 1 --2- 172.21.15.49:0/3425312454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec100db0 0x7f2eec101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.856+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/3425312454 >> 172.21.15.49:0/3425312454 conn(0x7f2eec076600 msgr2=0x7f2eec076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:06.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.856+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/3425312454 shutdown_connections 2024-03-26T18:01:06.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.856+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/3425312454 wait complete. 2024-03-26T18:01:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.856+0000 7f2ef0e3e700 1 Processor -- start 2024-03-26T18:01:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.856+0000 7f2ef0e3e700 1 -- start start 2024-03-26T18:01:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ef0e3e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec100db0 0x7f2eec19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ef0e3e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eec101770 0x7f2eec19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ef0e3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eec107480 0x7f2eec1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ef0e3e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec112720 con 0x7f2eec101770 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ef0e3e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1125a0 con 0x7f2eec107480 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ef0e3e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1128a0 con 0x7f2eec100db0 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ee9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eec101770 0x7f2eec19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2eea59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec100db0 0x7f2eec19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ee9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eec101770 0x7f2eec19ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36504/0 (socket says 172.21.15.49:36504) 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ee9d9b700 1 -- 172.21.15.49:0/2960279058 learned_addr learned my addr 172.21.15.49:0/2960279058 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2eead9d700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eec107480 0x7f2eec1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ee9d9b700 1 -- 172.21.15.49:0/2960279058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec100db0 msgr2=0x7f2eec19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ee9d9b700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec100db0 0x7f2eec19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ee9d9b700 1 -- 172.21.15.49:0/2960279058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eec107480 msgr2=0x7f2eec1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ee9d9b700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eec107480 0x7f2eec1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ee9d9b700 1 -- 172.21.15.49:0/2960279058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec199020 con 0x7f2eec101770 2024-03-26T18:01:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2eead9d700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eec107480 0x7f2eec1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:06.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2ee9d9b700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eec101770 0x7f2eec19ec40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000e9a0 tx=0x7f2ee000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:06.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.857+0000 7f2edb7fe700 1 -- 172.21.15.49:0/2960279058 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000ca10 con 0x7f2eec101770 2024-03-26T18:01:06.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.858+0000 7f2edb7fe700 1 -- 172.21.15.49:0/2960279058 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee000cbb0 con 0x7f2eec101770 2024-03-26T18:01:06.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.858+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/2960279058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec199310 con 0x7f2eec101770 2024-03-26T18:01:06.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.858+0000 7f2edb7fe700 1 -- 172.21.15.49:0/2960279058 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0018680 con 0x7f2eec101770 2024-03-26T18:01:06.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.858+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/2960279058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec1a55d0 con 0x7f2eec101770 2024-03-26T18:01:06.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.858+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/2960279058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eec069070 con 0x7f2eec101770 2024-03-26T18:01:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.860+0000 7f2edb7fe700 1 -- 172.21.15.49:0/2960279058 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f2ee0010430 con 0x7f2eec101770 2024-03-26T18:01:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.860+0000 7f2edb7fe700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ed40a3040 0x7f2ed40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.860+0000 7f2edb7fe700 1 -- 172.21.15.49:0/2960279058 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f2ee00cff80 con 0x7f2eec101770 2024-03-26T18:01:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.860+0000 7f2eea59c700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ed40a3040 0x7f2ed40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.861+0000 7f2eea59c700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ed40a3040 0x7f2ed40a5500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2eec199a00 tx=0x7f2edc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:06.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:06.864+0000 7f2edb7fe700 1 -- 172.21.15.49:0/2960279058 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2ee009c910 con 0x7f2eec101770 2024-03-26T18:01:07.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.072+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/2960279058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2eec1a5800 con 0x7f2eec101770 2024-03-26T18:01:07.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.072+0000 7f2edb7fe700 1 -- 172.21.15.49:0/2960279058 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ee009c210 con 0x7f2eec101770 2024-03-26T18:01:07.054 INFO:teuthology.orchestra.run.smithi049.stdout:55834574927 2024-03-26T18:01:07.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.074+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/2960279058 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ed40a3040 msgr2=0x7f2ed40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:07.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.074+0000 7f2ef0e3e700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ed40a3040 0x7f2ed40a5500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2eec199a00 tx=0x7f2edc009260 comp rx=0 tx=0).stop 2024-03-26T18:01:07.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.074+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/2960279058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eec101770 msgr2=0x7f2eec19ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:07.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.074+0000 7f2ef0e3e700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eec101770 0x7f2eec19ec40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000e9a0 tx=0x7f2ee000ecb0 comp rx=0 tx=0).stop 2024-03-26T18:01:07.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.074+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/2960279058 shutdown_connections 2024-03-26T18:01:07.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.074+0000 7f2ef0e3e700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f2ed40a3040 0x7f2ed40a5500 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:07.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.074+0000 7f2ef0e3e700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2eec107480 0x7f2eec1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:07.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.074+0000 7f2ef0e3e700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec100db0 0x7f2eec19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:07.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.074+0000 7f2ef0e3e700 1 --2- 172.21.15.49:0/2960279058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2eec101770 0x7f2eec19ec40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:07.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.074+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/2960279058 >> 172.21.15.49:0/2960279058 conn(0x7f2eec076600 msgr2=0x7f2eec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:07.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.075+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/2960279058 shutdown_connections 2024-03-26T18:01:07.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:07.075+0000 7f2ef0e3e700 1 -- 172.21.15.49:0/2960279058 wait complete. 2024-03-26T18:01:07.358 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:07 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2960279058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T18:01:07.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:07 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2960279058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T18:01:07.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:07 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2960279058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T18:01:08.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888122 got 146028888127 for osd.4 2024-03-26T18:01:08.086 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:08.336 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:08.339 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:08 smithi049 ceph-mon[32331]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-26T18:01:08.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:08 smithi171 ceph-mon[36033]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-26T18:01:08.638 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:08 smithi115 ceph-mon[36174]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-26T18:01:09.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.107+0000 7f5a08c15700 1 -- 172.21.15.49:0/3911882510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a04108400 msgr2=0x7f5a041087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:09.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.107+0000 7f5a08c15700 1 --2- 172.21.15.49:0/3911882510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a04108400 0x7f5a041087e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009fc0 tx=0x7f59f400b330 comp rx=0 tx=0).stop 2024-03-26T18:01:09.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.107+0000 7f5a08c15700 1 -- 172.21.15.49:0/3911882510 shutdown_connections 2024-03-26T18:01:09.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.107+0000 7f5a08c15700 1 --2- 172.21.15.49:0/3911882510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0406aa00 0x7f5a0410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:09.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.107+0000 7f5a08c15700 1 --2- 172.21.15.49:0/3911882510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a04108400 0x7f5a041087e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:09.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.107+0000 7f5a08c15700 1 --2- 172.21.15.49:0/3911882510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a0406a060 0x7f5a0406a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:09.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.107+0000 7f5a08c15700 1 -- 172.21.15.49:0/3911882510 >> 172.21.15.49:0/3911882510 conn(0x7f5a040781e0 msgr2=0x7f5a040785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:09.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.107+0000 7f5a08c15700 1 -- 172.21.15.49:0/3911882510 shutdown_connections 2024-03-26T18:01:09.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.107+0000 7f5a08c15700 1 -- 172.21.15.49:0/3911882510 wait complete. 2024-03-26T18:01:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.108+0000 7f5a08c15700 1 Processor -- start 2024-03-26T18:01:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.108+0000 7f5a08c15700 1 -- start start 2024-03-26T18:01:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a08c15700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0406a060 0x7f5a04101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a08c15700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0406aa00 0x7f5a041018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a08c15700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a04108400 0x7f5a04101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a08c15700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5a04112640 con 0x7f5a0406a060 2024-03-26T18:01:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a08c15700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5a041124c0 con 0x7f5a04108400 2024-03-26T18:01:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a08c15700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5a041127c0 con 0x7f5a0406aa00 2024-03-26T18:01:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a0259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0406a060 0x7f5a04101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a02d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a04108400 0x7f5a04101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a02d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a04108400 0x7f5a04101e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57816/0 (socket says 172.21.15.49:57816) 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a02d9d700 1 -- 172.21.15.49:0/2771115963 learned_addr learned my addr 172.21.15.49:0/2771115963 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a01d9b700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0406aa00 0x7f5a041018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a02d9d700 1 -- 172.21.15.49:0/2771115963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0406aa00 msgr2=0x7f5a041018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a02d9d700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0406aa00 0x7f5a041018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a02d9d700 1 -- 172.21.15.49:0/2771115963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0406a060 msgr2=0x7f5a04101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a02d9d700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0406a060 0x7f5a04101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a02d9d700 1 -- 172.21.15.49:0/2771115963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a041a7000 con 0x7f5a04108400 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a01d9b700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0406aa00 0x7f5a041018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.109+0000 7f5a02d9d700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a04108400 0x7f5a04101e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00e940 tx=0x7f59fc00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.110+0000 7f59fb7fe700 1 -- 172.21.15.49:0/2771115963 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc004a30 con 0x7f5a04108400 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.110+0000 7f5a08c15700 1 -- 172.21.15.49:0/2771115963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a041a72f0 con 0x7f5a04108400 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.110+0000 7f59fb7fe700 1 -- 172.21.15.49:0/2771115963 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59fc005500 con 0x7f5a04108400 2024-03-26T18:01:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.110+0000 7f59fb7fe700 1 -- 172.21.15.49:0/2771115963 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00ce20 con 0x7f5a04108400 2024-03-26T18:01:09.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.110+0000 7f5a08c15700 1 -- 172.21.15.49:0/2771115963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a041a7830 con 0x7f5a04108400 2024-03-26T18:01:09.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.111+0000 7f5a08c15700 1 -- 172.21.15.49:0/2771115963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a04069070 con 0x7f5a04108400 2024-03-26T18:01:09.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.112+0000 7f59fb7fe700 1 -- 172.21.15.49:0/2771115963 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7f59fc044b80 con 0x7f5a04108400 2024-03-26T18:01:09.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.113+0000 7f59fb7fe700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f59f00a3040 0x7f59f00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:09.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.113+0000 7f5a0259c700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f59f00a3040 0x7f59f00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:09.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.113+0000 7f59fb7fe700 1 -- 172.21.15.49:0/2771115963 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f59fc0cfa40 con 0x7f5a04108400 2024-03-26T18:01:09.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.114+0000 7f5a0259c700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f59f00a3040 0x7f59f00a5500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009da0 tx=0x7f59f4006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:09.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.120+0000 7f59fb7fe700 1 -- 172.21.15.49:0/2771115963 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f59fc09c370 con 0x7f5a04108400 2024-03-26T18:01:09.318 INFO:teuthology.orchestra.run.smithi049.stdout:347892351005 2024-03-26T18:01:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.336+0000 7f5a08c15700 1 -- 172.21.15.49:0/2771115963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5a0402dc00 con 0x7f5a04108400 2024-03-26T18:01:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.336+0000 7f59fb7fe700 1 -- 172.21.15.49:0/2771115963 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f59fc09bc70 con 0x7f5a04108400 2024-03-26T18:01:09.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 -- 172.21.15.49:0/2771115963 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f59f00a3040 msgr2=0x7f59f00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:09.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f59f00a3040 0x7f59f00a5500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009da0 tx=0x7f59f4006000 comp rx=0 tx=0).stop 2024-03-26T18:01:09.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 -- 172.21.15.49:0/2771115963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a04108400 msgr2=0x7f5a04101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:09.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a04108400 0x7f5a04101e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00e940 tx=0x7f59fc00ec50 comp rx=0 tx=0).stop 2024-03-26T18:01:09.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 -- 172.21.15.49:0/2771115963 shutdown_connections 2024-03-26T18:01:09.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f59f00a3040 0x7f59f00a5500 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a04108400 0x7f5a04101e00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a0406a060 0x7f5a04101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 --2- 172.21.15.49:0/2771115963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0406aa00 0x7f5a041018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 -- 172.21.15.49:0/2771115963 >> 172.21.15.49:0/2771115963 conn(0x7f5a040781e0 msgr2=0x7f5a040ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 -- 172.21.15.49:0/2771115963 shutdown_connections 2024-03-26T18:01:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:09.338+0000 7f5a08c15700 1 -- 172.21.15.49:0/2771115963 wait complete. 2024-03-26T18:01:09.727 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738451 for osd.0 2024-03-26T18:01:09.727 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:10.412 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:10 smithi049 ceph-mon[32331]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-26T18:01:10.413 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:10 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2771115963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T18:01:10.487 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:10 smithi115 ceph-mon[36174]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-26T18:01:10.487 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:10 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2771115963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T18:01:10.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:10 smithi171 ceph-mon[36033]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-26T18:01:10.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:10 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2771115963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T18:01:11.360 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574927 for osd.1 2024-03-26T18:01:11.361 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:11.748 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462962 got 201863462965 for osd.6 2024-03-26T18:01:11.748 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:12.290 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:12 smithi049 ceph-mon[32331]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-26T18:01:12.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:12 smithi171 ceph-mon[36033]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-26T18:01:12.641 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037797 got 257698037802 for osd.8 2024-03-26T18:01:12.642 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:12.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:12 smithi115 ceph-mon[36174]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-26T18:01:12.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.868+0000 7fa96d545700 1 -- 172.21.15.49:0/2218546389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa968101c50 msgr2=0x7fa96810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:12.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.868+0000 7fa96d545700 1 --2- 172.21.15.49:0/2218546389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa968101c50 0x7fa96810e130 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa96000b0a0 tx=0x7fa96000b3b0 comp rx=0 tx=0).stop 2024-03-26T18:01:12.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.868+0000 7fa96d545700 1 -- 172.21.15.49:0/2218546389 shutdown_connections 2024-03-26T18:01:12.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.868+0000 7fa96d545700 1 --2- 172.21.15.49:0/2218546389 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa968101c50 0x7fa96810e130 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:12.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.868+0000 7fa96d545700 1 --2- 172.21.15.49:0/2218546389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9681072f0 0x7fa9681076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:12.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.868+0000 7fa96d545700 1 --2- 172.21.15.49:0/2218546389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9681012b0 0x7fa968101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:12.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.868+0000 7fa96d545700 1 -- 172.21.15.49:0/2218546389 >> 172.21.15.49:0/2218546389 conn(0x7fa9680781e0 msgr2=0x7fa9680785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:12.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.868+0000 7fa96d545700 1 -- 172.21.15.49:0/2218546389 shutdown_connections 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.868+0000 7fa96d545700 1 -- 172.21.15.49:0/2218546389 wait complete. 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.868+0000 7fa96d545700 1 Processor -- start 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa96d545700 1 -- start start 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa96d545700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9681012b0 0x7fa96819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa96d545700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa968101c50 0x7fa96819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa96d545700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9681072f0 0x7fa968198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa96d545700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa968111570 con 0x7fa9681012b0 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa96d545700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681113f0 con 0x7fa968101c50 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa96d545700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681116f0 con 0x7fa9681072f0 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa966ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9681012b0 0x7fa96819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa966ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9681012b0 0x7fa96819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47958/0 (socket says 172.21.15.49:47958) 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa966ffd700 1 -- 172.21.15.49:0/3240379528 learned_addr learned my addr 172.21.15.49:0/3240379528 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa9677fe700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9681072f0 0x7fa968198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa9667fc700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa968101c50 0x7fa96819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:12.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa966ffd700 1 -- 172.21.15.49:0/3240379528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9681072f0 msgr2=0x7fa968198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa966ffd700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9681072f0 0x7fa968198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa966ffd700 1 -- 172.21.15.49:0/3240379528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa968101c50 msgr2=0x7fa96819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa966ffd700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa968101c50 0x7fa96819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa966ffd700 1 -- 172.21.15.49:0/3240379528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9681990e0 con 0x7fa9681012b0 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.869+0000 7fa966ffd700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9681012b0 0x7fa96819e7c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e9a0 tx=0x7fa95800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.870+0000 7fa94ffff700 1 -- 172.21.15.49:0/3240379528 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95800ca10 con 0x7fa9681012b0 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.870+0000 7fa96d545700 1 -- 172.21.15.49:0/3240379528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9681993d0 con 0x7fa9681012b0 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.870+0000 7fa94ffff700 1 -- 172.21.15.49:0/3240379528 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95800cbb0 con 0x7fa9681012b0 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.870+0000 7fa94ffff700 1 -- 172.21.15.49:0/3240379528 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958018680 con 0x7fa9681012b0 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.870+0000 7fa96d545700 1 -- 172.21.15.49:0/3240379528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9681a5690 con 0x7fa9681012b0 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.871+0000 7fa96d545700 1 -- 172.21.15.49:0/3240379528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa968069070 con 0x7fa9681012b0 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.872+0000 7fa94ffff700 1 -- 172.21.15.49:0/3240379528 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) ==== 137905+0+0 (secure 0 0 0) 0x7fa958010430 con 0x7fa9681012b0 2024-03-26T18:01:12.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.873+0000 7fa94ffff700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa9500a3040 0x7fa9500a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:12.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.873+0000 7fa9667fc700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa9500a3040 0x7fa9500a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:12.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.873+0000 7fa94ffff700 1 -- 172.21.15.49:0/3240379528 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fa9580cff30 con 0x7fa9681012b0 2024-03-26T18:01:12.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.873+0000 7fa9667fc700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa9500a3040 0x7fa9500a5500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9680787e0 tx=0x7fa95c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:12.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:12.876+0000 7fa94ffff700 1 -- 172.21.15.49:0/3240379528 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa95809c8c0 con 0x7fa9681012b0 2024-03-26T18:01:13.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.086+0000 7fa96d545700 1 -- 172.21.15.49:0/3240379528 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa9681a58c0 con 0x7fa9681012b0 2024-03-26T18:01:13.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.086+0000 7fa94ffff700 1 -- 172.21.15.49:0/3240379528 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa95809c1c0 con 0x7fa9681012b0 2024-03-26T18:01:13.068 INFO:teuthology.orchestra.run.smithi049.stdout:167503724603 2024-03-26T18:01:13.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.088+0000 7fa96d545700 1 -- 172.21.15.49:0/3240379528 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa9500a3040 msgr2=0x7fa9500a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.088+0000 7fa96d545700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa9500a3040 0x7fa9500a5500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9680787e0 tx=0x7fa95c009260 comp rx=0 tx=0).stop 2024-03-26T18:01:13.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.088+0000 7fa96d545700 1 -- 172.21.15.49:0/3240379528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9681012b0 msgr2=0x7fa96819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.088+0000 7fa96d545700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9681012b0 0x7fa96819e7c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e9a0 tx=0x7fa95800ecb0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.089+0000 7fa96d545700 1 -- 172.21.15.49:0/3240379528 shutdown_connections 2024-03-26T18:01:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.089+0000 7fa96d545700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa9500a3040 0x7fa9500a5500 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.089+0000 7fa96d545700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9681072f0 0x7fa968198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.089+0000 7fa96d545700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9681012b0 0x7fa96819e7c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.089+0000 7fa96d545700 1 --2- 172.21.15.49:0/3240379528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa968101c50 0x7fa96819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.089+0000 7fa96d545700 1 -- 172.21.15.49:0/3240379528 >> 172.21.15.49:0/3240379528 conn(0x7fa9680781e0 msgr2=0x7fa968100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.089+0000 7fa96d545700 1 -- 172.21.15.49:0/3240379528 shutdown_connections 2024-03-26T18:01:13.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:13.089+0000 7fa96d545700 1 -- 172.21.15.49:0/3240379528 wait complete. 2024-03-26T18:01:13.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:13 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3240379528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T18:01:13.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:13 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3240379528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T18:01:13.739 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:13 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3240379528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T18:01:13.745 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:13.774 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350998 got 347892351005 for osd.11 2024-03-26T18:01:13.774 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:14.537 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:14 smithi049 ceph-mon[32331]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-26T18:01:14.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:14 smithi171 ceph-mon[36033]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-26T18:01:14.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:14 smithi115 ceph-mon[36174]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-26T18:01:15.027 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724598 got 167503724603 for osd.5 2024-03-26T18:01:15.027 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:15.461 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:15 smithi049 ceph-mon[32331]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T18:01:15.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:15 smithi171 ceph-mon[36033]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T18:01:15.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.726+0000 7faf3047f700 1 -- 172.21.15.49:0/826989877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf28108420 msgr2=0x7faf28108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:15.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.726+0000 7faf3047f700 1 --2- 172.21.15.49:0/826989877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf28108420 0x7faf28108800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faf240099c0 tx=0x7faf24009cd0 comp rx=0 tx=0).stop 2024-03-26T18:01:15.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.726+0000 7faf3047f700 1 -- 172.21.15.49:0/826989877 shutdown_connections 2024-03-26T18:01:15.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.726+0000 7faf3047f700 1 --2- 172.21.15.49:0/826989877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf2806b470 0x7faf2810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:15.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.726+0000 7faf3047f700 1 --2- 172.21.15.49:0/826989877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf28108420 0x7faf28108800 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:15.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.726+0000 7faf3047f700 1 --2- 172.21.15.49:0/826989877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf2806aab0 0x7faf2806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:15.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.726+0000 7faf3047f700 1 -- 172.21.15.49:0/826989877 >> 172.21.15.49:0/826989877 conn(0x7faf28077300 msgr2=0x7faf28077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:15.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.726+0000 7faf3047f700 1 -- 172.21.15.49:0/826989877 shutdown_connections 2024-03-26T18:01:15.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.726+0000 7faf3047f700 1 -- 172.21.15.49:0/826989877 wait complete. 2024-03-26T18:01:15.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.727+0000 7faf3047f700 1 Processor -- start 2024-03-26T18:01:15.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.727+0000 7faf3047f700 1 -- start start 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.727+0000 7faf3047f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf2806aab0 0x7faf2819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.727+0000 7faf3047f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf2806b470 0x7faf2819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.727+0000 7faf3047f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf28108420 0x7faf28198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.727+0000 7faf3047f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faf281126a0 con 0x7faf2806aab0 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.727+0000 7faf3047f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf28112520 con 0x7faf2806b470 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.727+0000 7faf3047f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faf28112820 con 0x7faf28108420 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2e21b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf2806aab0 0x7faf2819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2da1a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf2806b470 0x7faf2819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2e21b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf2806aab0 0x7faf2819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47976/0 (socket says 172.21.15.49:47976) 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2da1a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf2806b470 0x7faf2819ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57852/0 (socket says 172.21.15.49:57852) 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2e21b700 1 -- 172.21.15.49:0/4094312173 learned_addr learned my addr 172.21.15.49:0/4094312173 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2ea1c700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf28108420 0x7faf28198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2da1a700 1 -- 172.21.15.49:0/4094312173 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf28108420 msgr2=0x7faf28198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2da1a700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf28108420 0x7faf28198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:15.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2da1a700 1 -- 172.21.15.49:0/4094312173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf2806aab0 msgr2=0x7faf2819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2da1a700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf2806aab0 0x7faf2819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2da1a700 1 -- 172.21.15.49:0/4094312173 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf28199060 con 0x7faf2806b470 2024-03-26T18:01:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2e21b700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf2806aab0 0x7faf2819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf2da1a700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf2806b470 0x7faf2819ec80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf1800e9f0 tx=0x7faf1800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf1f7fe700 1 -- 172.21.15.49:0/4094312173 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1800c9c0 con 0x7faf2806b470 2024-03-26T18:01:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf1f7fe700 1 -- 172.21.15.49:0/4094312173 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf1800cb60 con 0x7faf2806b470 2024-03-26T18:01:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf1f7fe700 1 -- 172.21.15.49:0/4094312173 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18018680 con 0x7faf2806b470 2024-03-26T18:01:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf3047f700 1 -- 172.21.15.49:0/4094312173 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf28199350 con 0x7faf2806b470 2024-03-26T18:01:15.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.728+0000 7faf3047f700 1 -- 172.21.15.49:0/4094312173 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf281a5610 con 0x7faf2806b470 2024-03-26T18:01:15.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.729+0000 7faf3047f700 1 -- 172.21.15.49:0/4094312173 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf28069070 con 0x7faf2806b470 2024-03-26T18:01:15.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.730+0000 7faf1f7fe700 1 -- 172.21.15.49:0/4094312173 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7faf18010430 con 0x7faf2806b470 2024-03-26T18:01:15.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.731+0000 7faf1f7fe700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7faf140a2fb0 0x7faf140a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:15.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.731+0000 7faf2e21b700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7faf140a2fb0 0x7faf140a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:15.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.731+0000 7faf1f7fe700 1 -- 172.21.15.49:0/4094312173 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7faf180cf8a0 con 0x7faf2806b470 2024-03-26T18:01:15.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.732+0000 7faf2e21b700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7faf140a2fb0 0x7faf140a5470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faf24009990 tx=0x7faf24005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:15.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.735+0000 7faf1f7fe700 1 -- 172.21.15.49:0/4094312173 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faf1809c330 con 0x7faf2806b470 2024-03-26T18:01:15.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:15 smithi115 ceph-mon[36174]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T18:01:15.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.944+0000 7faf3047f700 1 -- 172.21.15.49:0/4094312173 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7faf2802dc00 con 0x7faf2806b470 2024-03-26T18:01:15.927 INFO:teuthology.orchestra.run.smithi049.stdout:77309411404 2024-03-26T18:01:15.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.944+0000 7faf1f7fe700 1 -- 172.21.15.49:0/4094312173 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf1809bc30 con 0x7faf2806b470 2024-03-26T18:01:15.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 -- 172.21.15.49:0/4094312173 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7faf140a2fb0 msgr2=0x7faf140a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7faf140a2fb0 0x7faf140a5470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faf24009990 tx=0x7faf24005960 comp rx=0 tx=0).stop 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 -- 172.21.15.49:0/4094312173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf2806b470 msgr2=0x7faf2819ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf2806b470 0x7faf2819ec80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf1800e9f0 tx=0x7faf1800ed00 comp rx=0 tx=0).stop 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 -- 172.21.15.49:0/4094312173 shutdown_connections 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7faf140a2fb0 0x7faf140a5470 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf28108420 0x7faf28198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf2806aab0 0x7faf2819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 --2- 172.21.15.49:0/4094312173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf2806b470 0x7faf2819ec80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 -- 172.21.15.49:0/4094312173 >> 172.21.15.49:0/4094312173 conn(0x7faf28077300 msgr2=0x7faf280fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 -- 172.21.15.49:0/4094312173 shutdown_connections 2024-03-26T18:01:15.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:15.947+0000 7faf3047f700 1 -- 172.21.15.49:0/4094312173 wait complete. 2024-03-26T18:01:16.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.348+0000 7fc278a66700 1 -- 172.21.15.49:0/1226080028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2741065d0 msgr2=0x7fc27410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:16.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.348+0000 7fc278a66700 1 --2- 172.21.15.49:0/1226080028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2741065d0 0x7fc27410e1a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00b0a0 tx=0x7fc26c00b3b0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.348+0000 7fc278a66700 1 -- 172.21.15.49:0/1226080028 shutdown_connections 2024-03-26T18:01:16.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.348+0000 7fc278a66700 1 --2- 172.21.15.49:0/1226080028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2741065d0 0x7fc27410e1a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.348+0000 7fc278a66700 1 --2- 172.21.15.49:0/1226080028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc274101540 0x7fc274101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.348+0000 7fc278a66700 1 --2- 172.21.15.49:0/1226080028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc274101ef0 0x7fc274105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.348+0000 7fc278a66700 1 -- 172.21.15.49:0/1226080028 >> 172.21.15.49:0/1226080028 conn(0x7fc2740772d0 msgr2=0x7fc2740776e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:16.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.348+0000 7fc278a66700 1 -- 172.21.15.49:0/1226080028 shutdown_connections 2024-03-26T18:01:16.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.348+0000 7fc278a66700 1 -- 172.21.15.49:0/1226080028 wait complete. 2024-03-26T18:01:16.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc278a66700 1 Processor -- start 2024-03-26T18:01:16.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc278a66700 1 -- start start 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc278a66700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc274101540 0x7fc274199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc278a66700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc274101ef0 0x7fc274199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc278a66700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2741065d0 0x7fc2741a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc278a66700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741115b0 con 0x7fc274101ef0 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc278a66700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc274111430 con 0x7fc274101540 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc278a66700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc274111730 con 0x7fc2741065d0 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc27259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc274101540 0x7fc274199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc271d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc274101ef0 0x7fc274199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc27259c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc274101540 0x7fc274199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:57862/0 (socket says 172.21.15.49:57862) 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc27259c700 1 -- 172.21.15.49:0/2787993434 learned_addr learned my addr 172.21.15.49:0/2787993434 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:16.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.349+0000 7fc272d9d700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2741065d0 0x7fc2741a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc27259c700 1 -- 172.21.15.49:0/2787993434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2741065d0 msgr2=0x7fc2741a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc27259c700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2741065d0 0x7fc2741a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc27259c700 1 -- 172.21.15.49:0/2787993434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc274101ef0 msgr2=0x7fc274199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc27259c700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc274101ef0 0x7fc274199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc27259c700 1 -- 172.21.15.49:0/2787993434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2741a4f30 con 0x7fc274101540 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc271d9b700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc274101ef0 0x7fc274199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc272d9d700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2741065d0 0x7fc2741a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc27259c700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc274101540 0x7fc274199970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc26400e9c0 tx=0x7fc26400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc2637fe700 1 -- 172.21.15.49:0/2787993434 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26400cbd0 con 0x7fc274101540 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc278a66700 1 -- 172.21.15.49:0/2787993434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2741a5160 con 0x7fc274101540 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc2637fe700 1 -- 172.21.15.49:0/2787993434 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26400ee60 con 0x7fc274101540 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc2637fe700 1 -- 172.21.15.49:0/2787993434 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264005630 con 0x7fc274101540 2024-03-26T18:01:16.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.350+0000 7fc278a66700 1 -- 172.21.15.49:0/2787993434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2741a5650 con 0x7fc274101540 2024-03-26T18:01:16.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.351+0000 7fc278a66700 1 -- 172.21.15.49:0/2787993434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2741029d0 con 0x7fc274101540 2024-03-26T18:01:16.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.352+0000 7fc2637fe700 1 -- 172.21.15.49:0/2787993434 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fc264044b40 con 0x7fc274101540 2024-03-26T18:01:16.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.353+0000 7fc2637fe700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc25c0a2fb0 0x7fc25c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:16.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.353+0000 7fc2637fe700 1 -- 172.21.15.49:0/2787993434 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fc2640cf9a0 con 0x7fc274101540 2024-03-26T18:01:16.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.353+0000 7fc271d9b700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc25c0a2fb0 0x7fc25c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:16.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.353+0000 7fc271d9b700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc25c0a2fb0 0x7fc25c0a5470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc2741079c0 tx=0x7fc268009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:16.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.361+0000 7fc2637fe700 1 -- 172.21.15.49:0/2787993434 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc26409c490 con 0x7fc274101540 2024-03-26T18:01:16.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.565+0000 7fc278a66700 1 -- 172.21.15.49:0/2787993434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc27402dc00 con 0x7fc274101540 2024-03-26T18:01:16.548 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:16 smithi049 ceph-mon[32331]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-26T18:01:16.548 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:16 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4094312173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T18:01:16.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.565+0000 7fc2637fe700 1 -- 172.21.15.49:0/2787993434 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc26409bd90 con 0x7fc274101540 2024-03-26T18:01:16.549 INFO:teuthology.orchestra.run.smithi049.stdout:313532612643 2024-03-26T18:01:16.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.567+0000 7fc278a66700 1 -- 172.21.15.49:0/2787993434 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc25c0a2fb0 msgr2=0x7fc25c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:16.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc25c0a2fb0 0x7fc25c0a5470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc2741079c0 tx=0x7fc268009260 comp rx=0 tx=0).stop 2024-03-26T18:01:16.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 -- 172.21.15.49:0/2787993434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc274101540 msgr2=0x7fc274199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc274101540 0x7fc274199970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc26400e9c0 tx=0x7fc26400ecd0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 -- 172.21.15.49:0/2787993434 shutdown_connections 2024-03-26T18:01:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fc25c0a2fb0 0x7fc25c0a5470 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2741065d0 0x7fc2741a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc274101540 0x7fc274199970 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 --2- 172.21.15.49:0/2787993434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc274101ef0 0x7fc274199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 -- 172.21.15.49:0/2787993434 >> 172.21.15.49:0/2787993434 conn(0x7fc2740772d0 msgr2=0x7fc2740ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 -- 172.21.15.49:0/2787993434 shutdown_connections 2024-03-26T18:01:16.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:16.568+0000 7fc278a66700 1 -- 172.21.15.49:0/2787993434 wait complete. 2024-03-26T18:01:16.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:16 smithi171 ceph-mon[36033]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-26T18:01:16.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:16 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4094312173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T18:01:16.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:16 smithi115 ceph-mon[36174]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-26T18:01:16.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:16 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4094312173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T18:01:16.799 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411404 for osd.2 2024-03-26T18:01:16.799 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:17.502 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:17 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2787993434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T18:01:17.546 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612638 got 313532612643 for osd.10 2024-03-26T18:01:17.546 DEBUG:teuthology.parallel:result is None 2024-03-26T18:01:17.546 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-26T18:01:17.546 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T18:01:17.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:17 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2787993434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T18:01:17.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:17 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2787993434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T18:01:17.818 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:18.502 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:18 smithi049 ceph-mon[32331]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-26T18:01:18.502 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:18 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:01:18.502 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:18 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:01:18.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:18 smithi171 ceph-mon[36033]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-26T18:01:18.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:18 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:01:18.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:18 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:01:18.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:18 smithi115 ceph-mon[36174]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-26T18:01:18.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:18 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:01:18.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:18 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:01:19.294 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:01:19.294 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:01:19.294 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:19 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:01:19.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:01:19.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:01:19.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:19 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:01:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.653+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/3115854135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a581009f0 msgr2=0x7f8a58100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:19.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.653+0000 7f8a5dc2d700 1 --2- 172.21.15.49:0/3115854135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a581009f0 0x7f8a58100e70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009990 tx=0x7f8a48009ca0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.653+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/3115854135 shutdown_connections 2024-03-26T18:01:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.653+0000 7f8a5dc2d700 1 --2- 172.21.15.49:0/3115854135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a581013b0 0x7f8a5810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.653+0000 7f8a5dc2d700 1 --2- 172.21.15.49:0/3115854135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581069f0 0x7f8a58106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.653+0000 7f8a5dc2d700 1 --2- 172.21.15.49:0/3115854135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a581009f0 0x7f8a58100e70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.653+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/3115854135 >> 172.21.15.49:0/3115854135 conn(0x7f8a58077880 msgr2=0x7f8a580fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.653+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/3115854135 shutdown_connections 2024-03-26T18:01:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.653+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/3115854135 wait complete. 2024-03-26T18:01:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.654+0000 7f8a5dc2d700 1 Processor -- start 2024-03-26T18:01:19.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.654+0000 7f8a5dc2d700 1 -- start start 2024-03-26T18:01:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.654+0000 7f8a5dc2d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a581009f0 0x7f8a5819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.654+0000 7f8a5dc2d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581013b0 0x7f8a5819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.654+0000 7f8a5dc2d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a581069f0 0x7f8a581989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.654+0000 7f8a5dc2d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581104c0 con 0x7f8a581009f0 2024-03-26T18:01:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.654+0000 7f8a5dc2d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58110340 con 0x7f8a581013b0 2024-03-26T18:01:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.654+0000 7f8a5dc2d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58110640 con 0x7f8a581069f0 2024-03-26T18:01:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a56ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581013b0 0x7f8a5819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a577fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a581009f0 0x7f8a5819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a56ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581013b0 0x7f8a5819ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33092/0 (socket says 172.21.15.49:33092) 2024-03-26T18:01:19.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a56ffd700 1 -- 172.21.15.49:0/644597932 learned_addr learned my addr 172.21.15.49:0/644597932 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a57fff700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a581069f0 0x7f8a581989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a56ffd700 1 -- 172.21.15.49:0/644597932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a581069f0 msgr2=0x7f8a581989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a56ffd700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a581069f0 0x7f8a581989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a56ffd700 1 -- 172.21.15.49:0/644597932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a581009f0 msgr2=0x7f8a5819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a56ffd700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a581009f0 0x7f8a5819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a56ffd700 1 -- 172.21.15.49:0/644597932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a58199100 con 0x7f8a581013b0 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a577fe700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a581009f0 0x7f8a5819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a56ffd700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581013b0 0x7f8a5819ed20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8a48000c00 tx=0x7f8a4800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a54ff9700 1 -- 172.21.15.49:0/644597932 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48027070 con 0x7f8a581013b0 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a54ff9700 1 -- 172.21.15.49:0/644597932 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a48003d70 con 0x7f8a581013b0 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/644597932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a58199390 con 0x7f8a581013b0 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.655+0000 7f8a54ff9700 1 -- 172.21.15.49:0/644597932 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800f460 con 0x7f8a581013b0 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.656+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/644597932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a581a56b0 con 0x7f8a581013b0 2024-03-26T18:01:19.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.656+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/644597932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a58102130 con 0x7f8a581013b0 2024-03-26T18:01:19.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.658+0000 7f8a54ff9700 1 -- 172.21.15.49:0/644597932 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f8a4802a070 con 0x7f8a581013b0 2024-03-26T18:01:19.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.658+0000 7f8a54ff9700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8a400a2fb0 0x7f8a400a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:19.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.658+0000 7f8a577fe700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8a400a2fb0 0x7f8a400a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:19.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.658+0000 7f8a54ff9700 1 -- 172.21.15.49:0/644597932 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f8a480d08e0 con 0x7f8a581013b0 2024-03-26T18:01:19.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.659+0000 7f8a577fe700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8a400a2fb0 0x7f8a400a5470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a58199ae0 tx=0x7f8a4c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:19.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.662+0000 7f8a54ff9700 1 -- 172.21.15.49:0/644597932 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8a48013070 con 0x7f8a581013b0 2024-03-26T18:01:19.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:01:19.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:01:19.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:19 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' 2024-03-26T18:01:19.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.861+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/644597932 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a58199ed0 con 0x7f8a400a2fb0 2024-03-26T18:01:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.871+0000 7f8a54ff9700 1 -- 172.21.15.49:0/644597932 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340965 (secure 0 0 0) 0x7f8a4c070400 con 0x7f8a400a2fb0 2024-03-26T18:01:19.858 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:19.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.875+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/644597932 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8a400a2fb0 msgr2=0x7f8a400a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:19.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.875+0000 7f8a5dc2d700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8a400a2fb0 0x7f8a400a5470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a58199ae0 tx=0x7f8a4c010040 comp rx=0 tx=0).stop 2024-03-26T18:01:19.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.875+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/644597932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581013b0 msgr2=0x7f8a5819ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:19.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.875+0000 7f8a5dc2d700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581013b0 0x7f8a5819ed20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8a48000c00 tx=0x7f8a4800bab0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.875+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/644597932 shutdown_connections 2024-03-26T18:01:19.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.875+0000 7f8a5dc2d700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8a400a2fb0 0x7f8a400a5470 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.875+0000 7f8a5dc2d700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8a581069f0 0x7f8a581989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.875+0000 7f8a5dc2d700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a581009f0 0x7f8a5819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.875+0000 7f8a5dc2d700 1 --2- 172.21.15.49:0/644597932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a581013b0 0x7f8a5819ed20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:19.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.876+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/644597932 >> 172.21.15.49:0/644597932 conn(0x7f8a58077880 msgr2=0x7f8a580ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:19.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.876+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/644597932 shutdown_connections 2024-03-26T18:01:19.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:19.876+0000 7f8a5dc2d700 1 -- 172.21.15.49:0/644597932 wait complete. 2024-03-26T18:01:19.862 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-26T18:01:20.497 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-03-26T18:01:18.728138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660670,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":857,"num_read_kb":556,"num_write":435,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":859,"ondisk_log_size":859,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361544,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580280,"statfs":{"total":1151940427776,"available":1151570206720,"internally_reserved":0,"allocated":12689408,"data_stored":8003576,"data_compressed":361608,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":57,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003834"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.755931+0000","last_change":"2024-03-26T17:59:47.232887+0000","last_active":"2024-03-26T18:01:12.755931+0000","last_peered":"2024-03-26T18:01:12.755931+0000","last_clean":"2024-03-26T18:01:12.755931+0000","last_became_active":"2024-03-26T17:59:47.232559+0000","last_became_peered":"2024-03-26T17:59:47.232559+0000","last_unstale":"2024-03-26T18:01:12.755931+0000","last_undegraded":"2024-03-26T18:01:12.755931+0000","last_fullsized":"2024-03-26T18:01:12.755931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:29:01.759592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.749635+0000","last_change":"2024-03-26T17:59:47.230719+0000","last_active":"2024-03-26T18:01:12.749635+0000","last_peered":"2024-03-26T18:01:12.749635+0000","last_clean":"2024-03-26T18:01:12.749635+0000","last_became_active":"2024-03-26T17:59:47.230411+0000","last_became_peered":"2024-03-26T17:59:47.230411+0000","last_unstale":"2024-03-26T18:01:12.749635+0000","last_undegraded":"2024-03-26T18:01:12.749635+0000","last_fullsized":"2024-03-26T18:01:12.749635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:29:21.410798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.755633+0000","last_change":"2024-03-26T17:59:47.233625+0000","last_active":"2024-03-26T18:01:12.755633+0000","last_peered":"2024-03-26T18:01:12.755633+0000","last_clean":"2024-03-26T18:01:12.755633+0000","last_became_active":"2024-03-26T17:59:47.233318+0000","last_became_peered":"2024-03-26T17:59:47.233318+0000","last_unstale":"2024-03-26T18:01:12.755633+0000","last_undegraded":"2024-03-26T18:01:12.755633+0000","last_fullsized":"2024-03-26T18:01:12.755633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:53:08.524508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742180+0000","last_change":"2024-03-26T17:59:47.234380+0000","last_active":"2024-03-26T18:01:12.742180+0000","last_peered":"2024-03-26T18:01:12.742180+0000","last_clean":"2024-03-26T18:01:12.742180+0000","last_became_active":"2024-03-26T17:59:47.233804+0000","last_became_peered":"2024-03-26T17:59:47.233804+0000","last_unstale":"2024-03-26T18:01:12.742180+0000","last_undegraded":"2024-03-26T18:01:12.742180+0000","last_fullsized":"2024-03-26T18:01:12.742180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:26:59.956704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.736646+0000","last_change":"2024-03-26T17:59:47.232523+0000","last_active":"2024-03-26T18:01:12.736646+0000","last_peered":"2024-03-26T18:01:12.736646+0000","last_clean":"2024-03-26T18:01:12.736646+0000","last_became_active":"2024-03-26T17:59:47.232183+0000","last_became_peered":"2024-03-26T17:59:47.232183+0000","last_unstale":"2024-03-26T18:01:12.736646+0000","last_undegraded":"2024-03-26T18:01:12.736646+0000","last_fullsized":"2024-03-26T18:01:12.736646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:47:06.538092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.740919+0000","last_change":"2024-03-26T17:59:47.229078+0000","last_active":"2024-03-26T18:01:12.740919+0000","last_peered":"2024-03-26T18:01:12.740919+0000","last_clean":"2024-03-26T18:01:12.740919+0000","last_became_active":"2024-03-26T17:59:47.228757+0000","last_became_peered":"2024-03-26T17:59:47.228757+0000","last_unstale":"2024-03-26T18:01:12.740919+0000","last_undegraded":"2024-03-26T18:01:12.740919+0000","last_fullsized":"2024-03-26T18:01:12.740919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:41:32.459219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.736008+0000","last_change":"2024-03-26T17:59:47.232111+0000","last_active":"2024-03-26T18:01:12.736008+0000","last_peered":"2024-03-26T18:01:12.736008+0000","last_clean":"2024-03-26T18:01:12.736008+0000","last_became_active":"2024-03-26T17:59:47.231440+0000","last_became_peered":"2024-03-26T17:59:47.231440+0000","last_unstale":"2024-03-26T18:01:12.736008+0000","last_undegraded":"2024-03-26T18:01:12.736008+0000","last_fullsized":"2024-03-26T18:01:12.736008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:23:11.607084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743981+0000","last_change":"2024-03-26T17:59:47.231875+0000","last_active":"2024-03-26T18:01:12.743981+0000","last_peered":"2024-03-26T18:01:12.743981+0000","last_clean":"2024-03-26T18:01:12.743981+0000","last_became_active":"2024-03-26T17:59:47.231534+0000","last_became_peered":"2024-03-26T17:59:47.231534+0000","last_unstale":"2024-03-26T18:01:12.743981+0000","last_undegraded":"2024-03-26T18:01:12.743981+0000","last_fullsized":"2024-03-26T18:01:12.743981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:05:33.010733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.750378+0000","last_change":"2024-03-26T17:59:47.235321+0000","last_active":"2024-03-26T18:01:12.750378+0000","last_peered":"2024-03-26T18:01:12.750378+0000","last_clean":"2024-03-26T18:01:12.750378+0000","last_became_active":"2024-03-26T17:59:47.231778+0000","last_became_peered":"2024-03-26T17:59:47.231778+0000","last_unstale":"2024-03-26T18:01:12.750378+0000","last_undegraded":"2024-03-26T18:01:12.750378+0000","last_fullsized":"2024-03-26T18:01:12.750378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:58:25.613249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.740991+0000","last_change":"2024-03-26T17:59:47.224272+0000","last_active":"2024-03-26T18:01:12.740991+0000","last_peered":"2024-03-26T18:01:12.740991+0000","last_clean":"2024-03-26T18:01:12.740991+0000","last_became_active":"2024-03-26T17:59:47.223439+0000","last_became_peered":"2024-03-26T17:59:47.223439+0000","last_unstale":"2024-03-26T18:01:12.740991+0000","last_undegraded":"2024-03-26T18:01:12.740991+0000","last_fullsized":"2024-03-26T18:01:12.740991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:28:40.428780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.758466+0000","last_change":"2024-03-26T17:59:47.227589+0000","last_active":"2024-03-26T18:01:12.758466+0000","last_peered":"2024-03-26T18:01:12.758466+0000","last_clean":"2024-03-26T18:01:12.758466+0000","last_became_active":"2024-03-26T17:59:47.227134+0000","last_became_peered":"2024-03-26T17:59:47.227134+0000","last_unstale":"2024-03-26T18:01:12.758466+0000","last_undegraded":"2024-03-26T18:01:12.758466+0000","last_fullsized":"2024-03-26T18:01:12.758466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:39:33.093899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742069+0000","last_change":"2024-03-26T17:59:47.233329+0000","last_active":"2024-03-26T18:01:12.742069+0000","last_peered":"2024-03-26T18:01:12.742069+0000","last_clean":"2024-03-26T18:01:12.742069+0000","last_became_active":"2024-03-26T17:59:47.233039+0000","last_became_peered":"2024-03-26T17:59:47.233039+0000","last_unstale":"2024-03-26T18:01:12.742069+0000","last_undegraded":"2024-03-26T18:01:12.742069+0000","last_fullsized":"2024-03-26T18:01:12.742069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:35:43.280834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.744599+0000","last_change":"2024-03-26T17:59:47.234468+0000","last_active":"2024-03-26T18:01:12.744599+0000","last_peered":"2024-03-26T18:01:12.744599+0000","last_clean":"2024-03-26T18:01:12.744599+0000","last_became_active":"2024-03-26T17:59:47.234063+0000","last_became_peered":"2024-03-26T17:59:47.234063+0000","last_unstale":"2024-03-26T18:01:12.744599+0000","last_undegraded":"2024-03-26T18:01:12.744599+0000","last_fullsized":"2024-03-26T18:01:12.744599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:25.542893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.749644+0000","last_change":"2024-03-26T17:59:47.229076+0000","last_active":"2024-03-26T18:01:12.749644+0000","last_peered":"2024-03-26T18:01:12.749644+0000","last_clean":"2024-03-26T18:01:12.749644+0000","last_became_active":"2024-03-26T17:59:47.228719+0000","last_became_peered":"2024-03-26T17:59:47.228719+0000","last_unstale":"2024-03-26T18:01:12.749644+0000","last_undegraded":"2024-03-26T18:01:12.749644+0000","last_fullsized":"2024-03-26T18:01:12.749644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:03:15.085812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.752146+0000","last_change":"2024-03-26T17:59:47.239166+0000","last_active":"2024-03-26T18:01:12.752146+0000","last_peered":"2024-03-26T18:01:12.752146+0000","last_clean":"2024-03-26T18:01:12.752146+0000","last_became_active":"2024-03-26T17:59:47.236051+0000","last_became_peered":"2024-03-26T17:59:47.236051+0000","last_unstale":"2024-03-26T18:01:12.752146+0000","last_undegraded":"2024-03-26T18:01:12.752146+0000","last_fullsized":"2024-03-26T18:01:12.752146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:46:36.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.760702+0000","last_change":"2024-03-26T17:59:47.231246+0000","last_active":"2024-03-26T18:01:12.760702+0000","last_peered":"2024-03-26T18:01:12.760702+0000","last_clean":"2024-03-26T18:01:12.760702+0000","last_became_active":"2024-03-26T17:59:47.228587+0000","last_became_peered":"2024-03-26T17:59:47.228587+0000","last_unstale":"2024-03-26T18:01:12.760702+0000","last_undegraded":"2024-03-26T18:01:12.760702+0000","last_fullsized":"2024-03-26T18:01:12.760702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:02:25.850361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.756052+0000","last_change":"2024-03-26T17:59:47.234790+0000","last_active":"2024-03-26T18:01:12.756052+0000","last_peered":"2024-03-26T18:01:12.756052+0000","last_clean":"2024-03-26T18:01:12.756052+0000","last_became_active":"2024-03-26T17:59:47.234436+0000","last_became_peered":"2024-03-26T17:59:47.234436+0000","last_unstale":"2024-03-26T18:01:12.756052+0000","last_undegraded":"2024-03-26T18:01:12.756052+0000","last_fullsized":"2024-03-26T18:01:12.756052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:54:26.949089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.739425+0000","last_change":"2024-03-26T17:59:47.238064+0000","last_active":"2024-03-26T18:01:12.739425+0000","last_peered":"2024-03-26T18:01:12.739425+0000","last_clean":"2024-03-26T18:01:12.739425+0000","last_became_active":"2024-03-26T17:59:47.237754+0000","last_became_peered":"2024-03-26T17:59:47.237754+0000","last_unstale":"2024-03-26T18:01:12.739425+0000","last_undegraded":"2024-03-26T18:01:12.739425+0000","last_fullsized":"2024-03-26T18:01:12.739425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:27:29.603516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.735843+0000","last_change":"2024-03-26T18:00:44.593818+0000","last_active":"2024-03-26T18:01:12.735843+0000","last_peered":"2024-03-26T18:01:12.735843+0000","last_clean":"2024-03-26T18:01:12.735843+0000","last_became_active":"2024-03-26T18:00:44.593418+0000","last_became_peered":"2024-03-26T18:00:44.593418+0000","last_unstale":"2024-03-26T18:01:12.735843+0000","last_undegraded":"2024-03-26T18:01:12.735843+0000","last_fullsized":"2024-03-26T18:01:12.735843+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:31:09.913943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581910+0000","last_change":"2024-03-26T17:59:51.480711+0000","last_active":"2024-03-26T18:00:44.581910+0000","last_peered":"2024-03-26T18:00:44.581910+0000","last_clean":"2024-03-26T18:00:44.581910+0000","last_became_active":"2024-03-26T17:59:51.480111+0000","last_became_peered":"2024-03-26T17:59:51.480111+0000","last_unstale":"2024-03-26T18:00:44.581910+0000","last_undegraded":"2024-03-26T18:00:44.581910+0000","last_fullsized":"2024-03-26T18:00:44.581910+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:40:55.456138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.529259+0000","last_change":"2024-03-26T17:59:46.434642+0000","last_active":"2024-03-26T18:01:12.529259+0000","last_peered":"2024-03-26T18:01:12.529259+0000","last_clean":"2024-03-26T18:01:12.529259+0000","last_became_active":"2024-03-26T17:59:46.434209+0000","last_became_peered":"2024-03-26T17:59:46.434209+0000","last_unstale":"2024-03-26T18:01:12.529259+0000","last_undegraded":"2024-03-26T18:01:12.529259+0000","last_fullsized":"2024-03-26T18:01:12.529259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:55:05.365712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743383+0000","last_change":"2024-03-26T17:59:47.239356+0000","last_active":"2024-03-26T18:01:12.743383+0000","last_peered":"2024-03-26T18:01:12.743383+0000","last_clean":"2024-03-26T18:01:12.743383+0000","last_became_active":"2024-03-26T17:59:47.239020+0000","last_became_peered":"2024-03-26T17:59:47.239020+0000","last_unstale":"2024-03-26T18:01:12.743383+0000","last_undegraded":"2024-03-26T18:01:12.743383+0000","last_fullsized":"2024-03-26T18:01:12.743383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:56.992379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579466+0000","last_change":"2024-03-26T17:59:51.486894+0000","last_active":"2024-03-26T18:00:44.579466+0000","last_peered":"2024-03-26T18:00:44.579466+0000","last_clean":"2024-03-26T18:00:44.579466+0000","last_became_active":"2024-03-26T17:59:51.486465+0000","last_became_peered":"2024-03-26T17:59:51.486465+0000","last_unstale":"2024-03-26T18:00:44.579466+0000","last_undegraded":"2024-03-26T18:00:44.579466+0000","last_fullsized":"2024-03-26T18:00:44.579466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:15:10.145415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.582869+0000","last_change":"2024-03-26T17:59:46.437908+0000","last_active":"2024-03-26T18:00:44.582869+0000","last_peered":"2024-03-26T18:00:44.582869+0000","last_clean":"2024-03-26T18:00:44.582869+0000","last_became_active":"2024-03-26T17:59:46.437592+0000","last_became_peered":"2024-03-26T17:59:46.437592+0000","last_unstale":"2024-03-26T18:00:44.582869+0000","last_undegraded":"2024-03-26T18:00:44.582869+0000","last_fullsized":"2024-03-26T18:00:44.582869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:05.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.755854+0000","last_change":"2024-03-26T17:59:47.240291+0000","last_active":"2024-03-26T18:01:12.755854+0000","last_peered":"2024-03-26T18:01:12.755854+0000","last_clean":"2024-03-26T18:01:12.755854+0000","last_became_active":"2024-03-26T17:59:47.239893+0000","last_became_peered":"2024-03-26T17:59:47.239893+0000","last_unstale":"2024-03-26T18:01:12.755854+0000","last_undegraded":"2024-03-26T18:01:12.755854+0000","last_fullsized":"2024-03-26T18:01:12.755854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:44:22.946199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579736+0000","last_change":"2024-03-26T17:59:51.483970+0000","last_active":"2024-03-26T18:00:44.579736+0000","last_peered":"2024-03-26T18:00:44.579736+0000","last_clean":"2024-03-26T18:00:44.579736+0000","last_became_active":"2024-03-26T17:59:51.483585+0000","last_became_peered":"2024-03-26T17:59:51.483585+0000","last_unstale":"2024-03-26T18:00:44.579736+0000","last_undegraded":"2024-03-26T18:00:44.579736+0000","last_fullsized":"2024-03-26T18:00:44.579736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:22:30.873421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.580114+0000","last_change":"2024-03-26T17:59:46.439374+0000","last_active":"2024-03-26T18:00:44.580114+0000","last_peered":"2024-03-26T18:00:44.580114+0000","last_clean":"2024-03-26T18:00:44.580114+0000","last_became_active":"2024-03-26T17:59:46.439065+0000","last_became_peered":"2024-03-26T17:59:46.439065+0000","last_unstale":"2024-03-26T18:00:44.580114+0000","last_undegraded":"2024-03-26T18:00:44.580114+0000","last_fullsized":"2024-03-26T18:00:44.580114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:45:37.954375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.755201+0000","last_change":"2024-03-26T17:59:47.237168+0000","last_active":"2024-03-26T18:01:12.755201+0000","last_peered":"2024-03-26T18:01:12.755201+0000","last_clean":"2024-03-26T18:01:12.755201+0000","last_became_active":"2024-03-26T17:59:47.232573+0000","last_became_peered":"2024-03-26T17:59:47.232573+0000","last_unstale":"2024-03-26T18:01:12.755201+0000","last_undegraded":"2024-03-26T18:01:12.755201+0000","last_fullsized":"2024-03-26T18:01:12.755201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:00:30.236249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579657+0000","last_change":"2024-03-26T17:59:51.477500+0000","last_active":"2024-03-26T18:00:44.579657+0000","last_peered":"2024-03-26T18:00:44.579657+0000","last_clean":"2024-03-26T18:00:44.579657+0000","last_became_active":"2024-03-26T17:59:51.477064+0000","last_became_peered":"2024-03-26T17:59:51.477064+0000","last_unstale":"2024-03-26T18:00:44.579657+0000","last_undegraded":"2024-03-26T18:00:44.579657+0000","last_fullsized":"2024-03-26T18:00:44.579657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:07:05.265910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579399+0000","last_change":"2024-03-26T17:59:46.437738+0000","last_active":"2024-03-26T18:00:44.579399+0000","last_peered":"2024-03-26T18:00:44.579399+0000","last_clean":"2024-03-26T18:00:44.579399+0000","last_became_active":"2024-03-26T17:59:46.437373+0000","last_became_peered":"2024-03-26T17:59:46.437373+0000","last_unstale":"2024-03-26T18:00:44.579399+0000","last_undegraded":"2024-03-26T18:00:44.579399+0000","last_fullsized":"2024-03-26T18:00:44.579399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:49:32.700437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.741852+0000","last_change":"2024-03-26T18:00:44.593603+0000","last_active":"2024-03-26T18:01:12.741852+0000","last_peered":"2024-03-26T18:01:12.741852+0000","last_clean":"2024-03-26T18:01:12.741852+0000","last_became_active":"2024-03-26T18:00:44.593218+0000","last_became_peered":"2024-03-26T18:00:44.593218+0000","last_unstale":"2024-03-26T18:01:12.741852+0000","last_undegraded":"2024-03-26T18:01:12.741852+0000","last_fullsized":"2024-03-26T18:01:12.741852+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:16:46.424715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.582262+0000","last_change":"2024-03-26T17:59:51.479332+0000","last_active":"2024-03-26T18:00:44.582262+0000","last_peered":"2024-03-26T18:00:44.582262+0000","last_clean":"2024-03-26T18:00:44.582262+0000","last_became_active":"2024-03-26T17:59:51.478857+0000","last_became_peered":"2024-03-26T17:59:51.478857+0000","last_unstale":"2024-03-26T18:00:44.582262+0000","last_undegraded":"2024-03-26T18:00:44.582262+0000","last_fullsized":"2024-03-26T18:00:44.582262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:04:30.568741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:13.516508+0000","last_change":"2024-03-26T17:59:46.438017+0000","last_active":"2024-03-26T18:01:13.516508+0000","last_peered":"2024-03-26T18:01:13.516508+0000","last_clean":"2024-03-26T18:01:13.516508+0000","last_became_active":"2024-03-26T17:59:46.437625+0000","last_became_peered":"2024-03-26T17:59:46.437625+0000","last_unstale":"2024-03-26T18:01:13.516508+0000","last_undegraded":"2024-03-26T18:01:13.516508+0000","last_fullsized":"2024-03-26T18:01:13.516508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:33:27.460626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.737565+0000","last_change":"2024-03-26T17:59:47.232764+0000","last_active":"2024-03-26T18:01:12.737565+0000","last_peered":"2024-03-26T18:01:12.737565+0000","last_clean":"2024-03-26T18:01:12.737565+0000","last_became_active":"2024-03-26T17:59:47.231672+0000","last_became_peered":"2024-03-26T17:59:47.231672+0000","last_unstale":"2024-03-26T18:01:12.737565+0000","last_undegraded":"2024-03-26T18:01:12.737565+0000","last_fullsized":"2024-03-26T18:01:12.737565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.812245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.593797+0000","last_change":"2024-03-26T18:00:44.593797+0000","last_active":"2024-03-26T18:00:44.593797+0000","last_peered":"2024-03-26T18:00:44.593797+0000","last_clean":"2024-03-26T18:00:44.593797+0000","last_became_active":"2024-03-26T18:00:44.593366+0000","last_became_peered":"2024-03-26T18:00:44.593366+0000","last_unstale":"2024-03-26T18:00:44.593797+0000","last_undegraded":"2024-03-26T18:00:44.593797+0000","last_fullsized":"2024-03-26T18:00:44.593797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:13:47.901554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.592843+0000","last_change":"2024-03-26T17:59:46.438232+0000","last_active":"2024-03-26T18:00:44.592843+0000","last_peered":"2024-03-26T18:00:44.592843+0000","last_clean":"2024-03-26T18:00:44.592843+0000","last_became_active":"2024-03-26T17:59:46.437871+0000","last_became_peered":"2024-03-26T17:59:46.437871+0000","last_unstale":"2024-03-26T18:00:44.592843+0000","last_undegraded":"2024-03-26T18:00:44.592843+0000","last_fullsized":"2024-03-26T18:00:44.592843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:08.626264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.739433+0000","last_change":"2024-03-26T17:59:47.237144+0000","last_active":"2024-03-26T18:01:12.739433+0000","last_peered":"2024-03-26T18:01:12.739433+0000","last_clean":"2024-03-26T18:01:12.739433+0000","last_became_active":"2024-03-26T17:59:47.236696+0000","last_became_peered":"2024-03-26T17:59:47.236696+0000","last_unstale":"2024-03-26T18:01:12.739433+0000","last_undegraded":"2024-03-26T18:01:12.739433+0000","last_fullsized":"2024-03-26T18:01:12.739433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:49:31.197898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583290+0000","last_change":"2024-03-26T17:59:51.490204+0000","last_active":"2024-03-26T18:00:44.583290+0000","last_peered":"2024-03-26T18:00:44.583290+0000","last_clean":"2024-03-26T18:00:44.583290+0000","last_became_active":"2024-03-26T17:59:51.489846+0000","last_became_peered":"2024-03-26T17:59:51.489846+0000","last_unstale":"2024-03-26T18:00:44.583290+0000","last_undegraded":"2024-03-26T18:00:44.583290+0000","last_fullsized":"2024-03-26T18:00:44.583290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:26:12.284436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.592836+0000","last_change":"2024-03-26T17:59:46.429699+0000","last_active":"2024-03-26T18:00:44.592836+0000","last_peered":"2024-03-26T18:00:44.592836+0000","last_clean":"2024-03-26T18:00:44.592836+0000","last_became_active":"2024-03-26T17:59:46.429319+0000","last_became_peered":"2024-03-26T17:59:46.429319+0000","last_unstale":"2024-03-26T18:00:44.592836+0000","last_undegraded":"2024-03-26T18:00:44.592836+0000","last_fullsized":"2024-03-26T18:00:44.592836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:05:15.695696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743260+0000","last_change":"2024-03-26T17:59:47.239705+0000","last_active":"2024-03-26T18:01:12.743260+0000","last_peered":"2024-03-26T18:01:12.743260+0000","last_clean":"2024-03-26T18:01:12.743260+0000","last_became_active":"2024-03-26T17:59:47.239046+0000","last_became_peered":"2024-03-26T17:59:47.239046+0000","last_unstale":"2024-03-26T18:01:12.743260+0000","last_undegraded":"2024-03-26T18:01:12.743260+0000","last_fullsized":"2024-03-26T18:01:12.743260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:08:34.930460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583964+0000","last_change":"2024-03-26T17:59:51.477511+0000","last_active":"2024-03-26T18:00:44.583964+0000","last_peered":"2024-03-26T18:00:44.583964+0000","last_clean":"2024-03-26T18:00:44.583964+0000","last_became_active":"2024-03-26T17:59:51.473949+0000","last_became_peered":"2024-03-26T17:59:51.473949+0000","last_unstale":"2024-03-26T18:00:44.583964+0000","last_undegraded":"2024-03-26T18:00:44.583964+0000","last_fullsized":"2024-03-26T18:00:44.583964+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:26:01.431977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.585193+0000","last_change":"2024-03-26T17:59:46.433235+0000","last_active":"2024-03-26T18:00:44.585193+0000","last_peered":"2024-03-26T18:00:44.585193+0000","last_clean":"2024-03-26T18:00:44.585193+0000","last_became_active":"2024-03-26T17:59:46.432929+0000","last_became_peered":"2024-03-26T17:59:46.432929+0000","last_unstale":"2024-03-26T18:00:44.585193+0000","last_undegraded":"2024-03-26T18:00:44.585193+0000","last_fullsized":"2024-03-26T18:00:44.585193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:46:23.064447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.745202+0000","last_change":"2024-03-26T17:59:47.232526+0000","last_active":"2024-03-26T18:01:12.745202+0000","last_peered":"2024-03-26T18:01:12.745202+0000","last_clean":"2024-03-26T18:01:12.745202+0000","last_became_active":"2024-03-26T17:59:47.232216+0000","last_became_peered":"2024-03-26T17:59:47.232216+0000","last_unstale":"2024-03-26T18:01:12.745202+0000","last_undegraded":"2024-03-26T18:01:12.745202+0000","last_fullsized":"2024-03-26T18:01:12.745202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:20:07.510565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581975+0000","last_change":"2024-03-26T17:59:51.482706+0000","last_active":"2024-03-26T18:00:44.581975+0000","last_peered":"2024-03-26T18:00:44.581975+0000","last_clean":"2024-03-26T18:00:44.581975+0000","last_became_active":"2024-03-26T17:59:51.482312+0000","last_became_peered":"2024-03-26T17:59:51.482312+0000","last_unstale":"2024-03-26T18:00:44.581975+0000","last_undegraded":"2024-03-26T18:00:44.581975+0000","last_fullsized":"2024-03-26T18:00:44.581975+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:42:26.614170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:14.516917+0000","last_change":"2024-03-26T17:59:46.440919+0000","last_active":"2024-03-26T18:01:14.516917+0000","last_peered":"2024-03-26T18:01:14.516917+0000","last_clean":"2024-03-26T18:01:14.516917+0000","last_became_active":"2024-03-26T17:59:46.440579+0000","last_became_peered":"2024-03-26T17:59:46.440579+0000","last_unstale":"2024-03-26T18:01:14.516917+0000","last_undegraded":"2024-03-26T18:01:14.516917+0000","last_fullsized":"2024-03-26T18:01:14.516917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:38:27.325478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.751826+0000","last_change":"2024-03-26T17:59:47.236251+0000","last_active":"2024-03-26T18:01:12.751826+0000","last_peered":"2024-03-26T18:01:12.751826+0000","last_clean":"2024-03-26T18:01:12.751826+0000","last_became_active":"2024-03-26T17:59:47.235941+0000","last_became_peered":"2024-03-26T17:59:47.235941+0000","last_unstale":"2024-03-26T18:01:12.751826+0000","last_undegraded":"2024-03-26T18:01:12.751826+0000","last_fullsized":"2024-03-26T18:01:12.751826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:50:34.821242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.578919+0000","last_change":"2024-03-26T17:59:51.480155+0000","last_active":"2024-03-26T18:00:44.578919+0000","last_peered":"2024-03-26T18:00:44.578919+0000","last_clean":"2024-03-26T18:00:44.578919+0000","last_became_active":"2024-03-26T17:59:51.479761+0000","last_became_peered":"2024-03-26T17:59:51.479761+0000","last_unstale":"2024-03-26T18:00:44.578919+0000","last_undegraded":"2024-03-26T18:00:44.578919+0000","last_fullsized":"2024-03-26T18:00:44.578919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:03:44.256586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583422+0000","last_change":"2024-03-26T17:59:46.436559+0000","last_active":"2024-03-26T18:00:44.583422+0000","last_peered":"2024-03-26T18:00:44.583422+0000","last_clean":"2024-03-26T18:00:44.583422+0000","last_became_active":"2024-03-26T17:59:46.436224+0000","last_became_peered":"2024-03-26T17:59:46.436224+0000","last_unstale":"2024-03-26T18:00:44.583422+0000","last_undegraded":"2024-03-26T18:00:44.583422+0000","last_fullsized":"2024-03-26T18:00:44.583422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:12:56.716278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.752019+0000","last_change":"2024-03-26T17:59:47.240266+0000","last_active":"2024-03-26T18:01:12.752019+0000","last_peered":"2024-03-26T18:01:12.752019+0000","last_clean":"2024-03-26T18:01:12.752019+0000","last_became_active":"2024-03-26T17:59:47.238847+0000","last_became_peered":"2024-03-26T17:59:47.238847+0000","last_unstale":"2024-03-26T18:01:12.752019+0000","last_undegraded":"2024-03-26T18:01:12.752019+0000","last_fullsized":"2024-03-26T18:01:12.752019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:47:36.325585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579951+0000","last_change":"2024-03-26T17:59:51.487207+0000","last_active":"2024-03-26T18:00:44.579951+0000","last_peered":"2024-03-26T18:00:44.579951+0000","last_clean":"2024-03-26T18:00:44.579951+0000","last_became_active":"2024-03-26T17:59:51.486351+0000","last_became_peered":"2024-03-26T17:59:51.486351+0000","last_unstale":"2024-03-26T18:00:44.579951+0000","last_undegraded":"2024-03-26T18:00:44.579951+0000","last_fullsized":"2024-03-26T18:00:44.579951+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:35:03.445677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.584497+0000","last_change":"2024-03-26T17:59:46.437839+0000","last_active":"2024-03-26T18:00:44.584497+0000","last_peered":"2024-03-26T18:00:44.584497+0000","last_clean":"2024-03-26T18:00:44.584497+0000","last_became_active":"2024-03-26T17:59:46.437515+0000","last_became_peered":"2024-03-26T17:59:46.437515+0000","last_unstale":"2024-03-26T18:00:44.584497+0000","last_undegraded":"2024-03-26T18:00:44.584497+0000","last_fullsized":"2024-03-26T18:00:44.584497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:38:34.875841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.755274+0000","last_change":"2024-03-26T17:59:47.234833+0000","last_active":"2024-03-26T18:01:12.755274+0000","last_peered":"2024-03-26T18:01:12.755274+0000","last_clean":"2024-03-26T18:01:12.755274+0000","last_became_active":"2024-03-26T17:59:47.232925+0000","last_became_peered":"2024-03-26T17:59:47.232925+0000","last_unstale":"2024-03-26T18:01:12.755274+0000","last_undegraded":"2024-03-26T18:01:12.755274+0000","last_fullsized":"2024-03-26T18:01:12.755274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:42:23.618396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579068+0000","last_change":"2024-03-26T17:59:51.481550+0000","last_active":"2024-03-26T18:00:44.579068+0000","last_peered":"2024-03-26T18:00:44.579068+0000","last_clean":"2024-03-26T18:00:44.579068+0000","last_became_active":"2024-03-26T17:59:51.481142+0000","last_became_peered":"2024-03-26T17:59:51.481142+0000","last_unstale":"2024-03-26T18:00:44.579068+0000","last_undegraded":"2024-03-26T18:00:44.579068+0000","last_fullsized":"2024-03-26T18:00:44.579068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:37.529859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'39","reported_seq":211,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:14.516782+0000","last_change":"2024-03-26T17:59:46.438936+0000","last_active":"2024-03-26T18:01:14.516782+0000","last_peered":"2024-03-26T18:01:14.516782+0000","last_clean":"2024-03-26T18:01:14.516782+0000","last_became_active":"2024-03-26T17:59:46.438609+0000","last_became_peered":"2024-03-26T17:59:46.438609+0000","last_unstale":"2024-03-26T18:01:14.516782+0000","last_undegraded":"2024-03-26T18:01:14.516782+0000","last_fullsized":"2024-03-26T18:01:14.516782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:38:36.333660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62582,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":145,"num_read_kb":134,"num_write":41,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.725607+0000","last_change":"2024-03-26T17:59:47.224495+0000","last_active":"2024-03-26T18:01:12.725607+0000","last_peered":"2024-03-26T18:01:12.725607+0000","last_clean":"2024-03-26T18:01:12.725607+0000","last_became_active":"2024-03-26T17:59:47.224184+0000","last_became_peered":"2024-03-26T17:59:47.224184+0000","last_unstale":"2024-03-26T18:01:12.725607+0000","last_undegraded":"2024-03-26T18:01:12.725607+0000","last_fullsized":"2024-03-26T18:01:12.725607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:17:23.028748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.584750+0000","last_change":"2024-03-26T17:59:51.483006+0000","last_active":"2024-03-26T18:00:44.584750+0000","last_peered":"2024-03-26T18:00:44.584750+0000","last_clean":"2024-03-26T18:00:44.584750+0000","last_became_active":"2024-03-26T17:59:51.482281+0000","last_became_peered":"2024-03-26T17:59:51.482281+0000","last_unstale":"2024-03-26T18:00:44.584750+0000","last_undegraded":"2024-03-26T18:00:44.584750+0000","last_fullsized":"2024-03-26T18:00:44.584750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:05:25.322809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583397+0000","last_change":"2024-03-26T17:59:46.440480+0000","last_active":"2024-03-26T18:00:44.583397+0000","last_peered":"2024-03-26T18:00:44.583397+0000","last_clean":"2024-03-26T18:00:44.583397+0000","last_became_active":"2024-03-26T17:59:46.439574+0000","last_became_peered":"2024-03-26T17:59:46.439574+0000","last_unstale":"2024-03-26T18:00:44.583397+0000","last_undegraded":"2024-03-26T18:00:44.583397+0000","last_fullsized":"2024-03-26T18:00:44.583397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:50:42.296440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.748020+0000","last_change":"2024-03-26T17:59:47.229944+0000","last_active":"2024-03-26T18:01:12.748020+0000","last_peered":"2024-03-26T18:01:12.748020+0000","last_clean":"2024-03-26T18:01:12.748020+0000","last_became_active":"2024-03-26T17:59:47.229626+0000","last_became_peered":"2024-03-26T17:59:47.229626+0000","last_unstale":"2024-03-26T18:01:12.748020+0000","last_undegraded":"2024-03-26T18:01:12.748020+0000","last_fullsized":"2024-03-26T18:01:12.748020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:18.998051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.596191+0000","last_change":"2024-03-26T18:00:44.596191+0000","last_active":"2024-03-26T18:00:44.596191+0000","last_peered":"2024-03-26T18:00:44.596191+0000","last_clean":"2024-03-26T18:00:44.596191+0000","last_became_active":"2024-03-26T18:00:44.595719+0000","last_became_peered":"2024-03-26T18:00:44.595719+0000","last_unstale":"2024-03-26T18:00:44.596191+0000","last_undegraded":"2024-03-26T18:00:44.596191+0000","last_fullsized":"2024-03-26T18:00:44.596191+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:50:17.967507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.585223+0000","last_change":"2024-03-26T17:59:46.435999+0000","last_active":"2024-03-26T18:00:44.585223+0000","last_peered":"2024-03-26T18:00:44.585223+0000","last_clean":"2024-03-26T18:00:44.585223+0000","last_became_active":"2024-03-26T17:59:46.435611+0000","last_became_peered":"2024-03-26T17:59:46.435611+0000","last_unstale":"2024-03-26T18:00:44.585223+0000","last_undegraded":"2024-03-26T18:00:44.585223+0000","last_fullsized":"2024-03-26T18:00:44.585223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:13:14.904792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.760591+0000","last_change":"2024-03-26T17:59:47.236395+0000","last_active":"2024-03-26T18:01:12.760591+0000","last_peered":"2024-03-26T18:01:12.760591+0000","last_clean":"2024-03-26T18:01:12.760591+0000","last_became_active":"2024-03-26T17:59:47.236055+0000","last_became_peered":"2024-03-26T17:59:47.236055+0000","last_unstale":"2024-03-26T18:01:12.760591+0000","last_undegraded":"2024-03-26T18:01:12.760591+0000","last_fullsized":"2024-03-26T18:01:12.760591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:06:26.131419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579523+0000","last_change":"2024-03-26T17:59:51.484584+0000","last_active":"2024-03-26T18:00:44.579523+0000","last_peered":"2024-03-26T18:00:44.579523+0000","last_clean":"2024-03-26T18:00:44.579523+0000","last_became_active":"2024-03-26T17:59:51.484138+0000","last_became_peered":"2024-03-26T17:59:51.484138+0000","last_unstale":"2024-03-26T18:00:44.579523+0000","last_undegraded":"2024-03-26T18:00:44.579523+0000","last_fullsized":"2024-03-26T18:00:44.579523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:49:40.163174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:15.525152+0000","last_change":"2024-03-26T17:59:46.433243+0000","last_active":"2024-03-26T18:01:15.525152+0000","last_peered":"2024-03-26T18:01:15.525152+0000","last_clean":"2024-03-26T18:01:15.525152+0000","last_became_active":"2024-03-26T17:59:46.432939+0000","last_became_peered":"2024-03-26T17:59:46.432939+0000","last_unstale":"2024-03-26T18:01:15.525152+0000","last_undegraded":"2024-03-26T18:01:15.525152+0000","last_fullsized":"2024-03-26T18:01:15.525152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:44:09.473862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.756403+0000","last_change":"2024-03-26T18:00:44.594670+0000","last_active":"2024-03-26T18:01:12.756403+0000","last_peered":"2024-03-26T18:01:12.756403+0000","last_clean":"2024-03-26T18:01:12.756403+0000","last_became_active":"2024-03-26T18:00:44.594329+0000","last_became_peered":"2024-03-26T18:00:44.594329+0000","last_unstale":"2024-03-26T18:01:12.756403+0000","last_undegraded":"2024-03-26T18:01:12.756403+0000","last_fullsized":"2024-03-26T18:01:12.756403+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:26:05.600410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.593287+0000","last_change":"2024-03-26T17:59:51.488010+0000","last_active":"2024-03-26T18:00:44.593287+0000","last_peered":"2024-03-26T18:00:44.593287+0000","last_clean":"2024-03-26T18:00:44.593287+0000","last_became_active":"2024-03-26T17:59:51.486406+0000","last_became_peered":"2024-03-26T17:59:51.486406+0000","last_unstale":"2024-03-26T18:00:44.593287+0000","last_undegraded":"2024-03-26T18:00:44.593287+0000","last_fullsized":"2024-03-26T18:00:44.593287+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:54:49.102720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579259+0000","last_change":"2024-03-26T17:59:46.438764+0000","last_active":"2024-03-26T18:00:44.579259+0000","last_peered":"2024-03-26T18:00:44.579259+0000","last_clean":"2024-03-26T18:00:44.579259+0000","last_became_active":"2024-03-26T17:59:46.438329+0000","last_became_peered":"2024-03-26T17:59:46.438329+0000","last_unstale":"2024-03-26T18:00:44.579259+0000","last_undegraded":"2024-03-26T18:00:44.579259+0000","last_fullsized":"2024-03-26T18:00:44.579259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:33:19.025134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.738879+0000","last_change":"2024-03-26T17:59:47.237020+0000","last_active":"2024-03-26T18:01:12.738879+0000","last_peered":"2024-03-26T18:01:12.738879+0000","last_clean":"2024-03-26T18:01:12.738879+0000","last_became_active":"2024-03-26T17:59:47.236693+0000","last_became_peered":"2024-03-26T17:59:47.236693+0000","last_unstale":"2024-03-26T18:01:12.738879+0000","last_undegraded":"2024-03-26T18:01:12.738879+0000","last_fullsized":"2024-03-26T18:01:12.738879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:40:25.473994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.584148+0000","last_change":"2024-03-26T17:59:51.482799+0000","last_active":"2024-03-26T18:00:44.584148+0000","last_peered":"2024-03-26T18:00:44.584148+0000","last_clean":"2024-03-26T18:00:44.584148+0000","last_became_active":"2024-03-26T17:59:51.482246+0000","last_became_peered":"2024-03-26T17:59:51.482246+0000","last_unstale":"2024-03-26T18:00:44.584148+0000","last_undegraded":"2024-03-26T18:00:44.584148+0000","last_fullsized":"2024-03-26T18:00:44.584148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:43:11.132952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579701+0000","last_change":"2024-03-26T17:59:46.438038+0000","last_active":"2024-03-26T18:00:44.579701+0000","last_peered":"2024-03-26T18:00:44.579701+0000","last_clean":"2024-03-26T18:00:44.579701+0000","last_became_active":"2024-03-26T17:59:46.437654+0000","last_became_peered":"2024-03-26T17:59:46.437654+0000","last_unstale":"2024-03-26T18:00:44.579701+0000","last_undegraded":"2024-03-26T18:00:44.579701+0000","last_fullsized":"2024-03-26T18:00:44.579701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:33:04.341028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.738114+0000","last_change":"2024-03-26T18:00:44.596184+0000","last_active":"2024-03-26T18:01:12.738114+0000","last_peered":"2024-03-26T18:01:12.738114+0000","last_clean":"2024-03-26T18:01:12.738114+0000","last_became_active":"2024-03-26T18:00:44.595766+0000","last_became_peered":"2024-03-26T18:00:44.595766+0000","last_unstale":"2024-03-26T18:01:12.738114+0000","last_undegraded":"2024-03-26T18:01:12.738114+0000","last_fullsized":"2024-03-26T18:01:12.738114+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:02:33.148661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579514+0000","last_change":"2024-03-26T17:59:51.477057+0000","last_active":"2024-03-26T18:00:44.579514+0000","last_peered":"2024-03-26T18:00:44.579514+0000","last_clean":"2024-03-26T18:00:44.579514+0000","last_became_active":"2024-03-26T17:59:51.476576+0000","last_became_peered":"2024-03-26T17:59:51.476576+0000","last_unstale":"2024-03-26T18:00:44.579514+0000","last_undegraded":"2024-03-26T18:00:44.579514+0000","last_fullsized":"2024-03-26T18:00:44.579514+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:42.533519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.587240+0000","last_change":"2024-03-26T18:00:44.587240+0000","last_active":"2024-03-26T18:00:44.587240+0000","last_peered":"2024-03-26T18:00:44.587240+0000","last_clean":"2024-03-26T18:00:44.587240+0000","last_became_active":"2024-03-26T18:00:44.586857+0000","last_became_peered":"2024-03-26T18:00:44.586857+0000","last_unstale":"2024-03-26T18:00:44.587240+0000","last_undegraded":"2024-03-26T18:00:44.587240+0000","last_fullsized":"2024-03-26T18:00:44.587240+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:28:02.590070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.738117+0000","last_change":"2024-03-26T17:59:47.232436+0000","last_active":"2024-03-26T18:01:12.738117+0000","last_peered":"2024-03-26T18:01:12.738117+0000","last_clean":"2024-03-26T18:01:12.738117+0000","last_became_active":"2024-03-26T17:59:47.232129+0000","last_became_peered":"2024-03-26T17:59:47.232129+0000","last_unstale":"2024-03-26T18:01:12.738117+0000","last_undegraded":"2024-03-26T18:01:12.738117+0000","last_fullsized":"2024-03-26T18:01:12.738117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:27:54.504624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.593148+0000","last_change":"2024-03-26T17:59:51.475972+0000","last_active":"2024-03-26T18:00:44.593148+0000","last_peered":"2024-03-26T18:00:44.593148+0000","last_clean":"2024-03-26T18:00:44.593148+0000","last_became_active":"2024-03-26T17:59:51.475531+0000","last_became_peered":"2024-03-26T17:59:51.475531+0000","last_unstale":"2024-03-26T18:00:44.593148+0000","last_undegraded":"2024-03-26T18:00:44.593148+0000","last_fullsized":"2024-03-26T18:00:44.593148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:43:23.639809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:16.534095+0000","last_change":"2024-03-26T17:59:46.432980+0000","last_active":"2024-03-26T18:01:16.534095+0000","last_peered":"2024-03-26T18:01:16.534095+0000","last_clean":"2024-03-26T18:01:16.534095+0000","last_became_active":"2024-03-26T17:59:46.432095+0000","last_became_peered":"2024-03-26T17:59:46.432095+0000","last_unstale":"2024-03-26T18:01:16.534095+0000","last_undegraded":"2024-03-26T18:01:16.534095+0000","last_fullsized":"2024-03-26T18:01:16.534095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:19:36.897555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.747649+0000","last_change":"2024-03-26T17:59:47.229623+0000","last_active":"2024-03-26T18:01:12.747649+0000","last_peered":"2024-03-26T18:01:12.747649+0000","last_clean":"2024-03-26T18:01:12.747649+0000","last_became_active":"2024-03-26T17:59:47.228781+0000","last_became_peered":"2024-03-26T17:59:47.228781+0000","last_unstale":"2024-03-26T18:01:12.747649+0000","last_undegraded":"2024-03-26T18:01:12.747649+0000","last_fullsized":"2024-03-26T18:01:12.747649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:54:22.171273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.593230+0000","last_change":"2024-03-26T17:59:51.483247+0000","last_active":"2024-03-26T18:00:44.593230+0000","last_peered":"2024-03-26T18:00:44.593230+0000","last_clean":"2024-03-26T18:00:44.593230+0000","last_became_active":"2024-03-26T17:59:51.482854+0000","last_became_peered":"2024-03-26T17:59:51.482854+0000","last_unstale":"2024-03-26T18:00:44.593230+0000","last_undegraded":"2024-03-26T18:00:44.593230+0000","last_fullsized":"2024-03-26T18:00:44.593230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:24:13.532986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.733522+0000","last_change":"2024-03-26T17:59:47.238866+0000","last_active":"2024-03-26T18:01:12.733522+0000","last_peered":"2024-03-26T18:01:12.733522+0000","last_clean":"2024-03-26T18:01:12.733522+0000","last_became_active":"2024-03-26T17:59:47.238366+0000","last_became_peered":"2024-03-26T17:59:47.238366+0000","last_unstale":"2024-03-26T18:01:12.733522+0000","last_undegraded":"2024-03-26T18:01:12.733522+0000","last_fullsized":"2024-03-26T18:01:12.733522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:04.045820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:15.524784+0000","last_change":"2024-03-26T17:59:46.433046+0000","last_active":"2024-03-26T18:01:15.524784+0000","last_peered":"2024-03-26T18:01:15.524784+0000","last_clean":"2024-03-26T18:01:15.524784+0000","last_became_active":"2024-03-26T17:59:46.432585+0000","last_became_peered":"2024-03-26T17:59:46.432585+0000","last_unstale":"2024-03-26T18:01:15.524784+0000","last_undegraded":"2024-03-26T18:01:15.524784+0000","last_fullsized":"2024-03-26T18:01:15.524784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:17.051521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583509+0000","last_change":"2024-03-26T17:59:51.478866+0000","last_active":"2024-03-26T18:00:44.583509+0000","last_peered":"2024-03-26T18:00:44.583509+0000","last_clean":"2024-03-26T18:00:44.583509+0000","last_became_active":"2024-03-26T17:59:51.478497+0000","last_became_peered":"2024-03-26T17:59:51.478497+0000","last_unstale":"2024-03-26T18:00:44.583509+0000","last_undegraded":"2024-03-26T18:00:44.583509+0000","last_fullsized":"2024-03-26T18:00:44.583509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:15:05.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.747008+0000","last_change":"2024-03-26T17:59:47.232349+0000","last_active":"2024-03-26T18:01:12.747008+0000","last_peered":"2024-03-26T18:01:12.747008+0000","last_clean":"2024-03-26T18:01:12.747008+0000","last_became_active":"2024-03-26T17:59:47.232001+0000","last_became_peered":"2024-03-26T17:59:47.232001+0000","last_unstale":"2024-03-26T18:01:12.747008+0000","last_undegraded":"2024-03-26T18:01:12.747008+0000","last_fullsized":"2024-03-26T18:01:12.747008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:42:14.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579850+0000","last_change":"2024-03-26T17:59:46.431522+0000","last_active":"2024-03-26T18:00:44.579850+0000","last_peered":"2024-03-26T18:00:44.579850+0000","last_clean":"2024-03-26T18:00:44.579850+0000","last_became_active":"2024-03-26T17:59:46.431201+0000","last_became_peered":"2024-03-26T17:59:46.431201+0000","last_unstale":"2024-03-26T18:00:44.579850+0000","last_undegraded":"2024-03-26T18:00:44.579850+0000","last_fullsized":"2024-03-26T18:00:44.579850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:11.298111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.582849+0000","last_change":"2024-03-26T17:58:38.167621+0000","last_active":"2024-03-26T18:00:44.582849+0000","last_peered":"2024-03-26T18:00:44.582849+0000","last_clean":"2024-03-26T18:00:44.582849+0000","last_became_active":"2024-03-26T17:58:38.167247+0000","last_became_peered":"2024-03-26T17:58:38.167247+0000","last_unstale":"2024-03-26T18:00:44.582849+0000","last_undegraded":"2024-03-26T18:00:44.582849+0000","last_fullsized":"2024-03-26T18:00:44.582849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:55:11.216723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:55:11.216723+0000","last_clean_scrub_stamp":"2024-03-26T17:55:11.216723+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:59:04.140225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.585109+0000","last_change":"2024-03-26T17:59:51.480354+0000","last_active":"2024-03-26T18:00:44.585109+0000","last_peered":"2024-03-26T18:00:44.585109+0000","last_clean":"2024-03-26T18:00:44.585109+0000","last_became_active":"2024-03-26T17:59:51.479474+0000","last_became_peered":"2024-03-26T17:59:51.479474+0000","last_unstale":"2024-03-26T18:00:44.585109+0000","last_undegraded":"2024-03-26T18:00:44.585109+0000","last_fullsized":"2024-03-26T18:00:44.585109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:48:31.846364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742931+0000","last_change":"2024-03-26T17:59:47.226036+0000","last_active":"2024-03-26T18:01:12.742931+0000","last_peered":"2024-03-26T18:01:12.742931+0000","last_clean":"2024-03-26T18:01:12.742931+0000","last_became_active":"2024-03-26T17:59:47.225722+0000","last_became_peered":"2024-03-26T17:59:47.225722+0000","last_unstale":"2024-03-26T18:01:12.742931+0000","last_undegraded":"2024-03-26T18:01:12.742931+0000","last_fullsized":"2024-03-26T18:01:12.742931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:28:39.310849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.582900+0000","last_change":"2024-03-26T17:59:46.432635+0000","last_active":"2024-03-26T18:00:44.582900+0000","last_peered":"2024-03-26T18:00:44.582900+0000","last_clean":"2024-03-26T18:00:44.582900+0000","last_became_active":"2024-03-26T17:59:46.432317+0000","last_became_peered":"2024-03-26T17:59:46.432317+0000","last_unstale":"2024-03-26T18:00:44.582900+0000","last_undegraded":"2024-03-26T18:00:44.582900+0000","last_fullsized":"2024-03-26T18:00:44.582900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:48:15.018906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581902+0000","last_change":"2024-03-26T17:59:51.480678+0000","last_active":"2024-03-26T18:00:44.581902+0000","last_peered":"2024-03-26T18:00:44.581902+0000","last_clean":"2024-03-26T18:00:44.581902+0000","last_became_active":"2024-03-26T17:59:51.480290+0000","last_became_peered":"2024-03-26T17:59:51.480290+0000","last_unstale":"2024-03-26T18:00:44.581902+0000","last_undegraded":"2024-03-26T18:00:44.581902+0000","last_fullsized":"2024-03-26T18:00:44.581902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:02:20.279963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.580026+0000","last_change":"2024-03-26T17:59:46.445569+0000","last_active":"2024-03-26T18:00:44.580026+0000","last_peered":"2024-03-26T18:00:44.580026+0000","last_clean":"2024-03-26T18:00:44.580026+0000","last_became_active":"2024-03-26T17:59:46.445114+0000","last_became_peered":"2024-03-26T17:59:46.445114+0000","last_unstale":"2024-03-26T18:00:44.580026+0000","last_undegraded":"2024-03-26T18:00:44.580026+0000","last_fullsized":"2024-03-26T18:00:44.580026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:18:59.367401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742871+0000","last_change":"2024-03-26T17:59:47.227901+0000","last_active":"2024-03-26T18:01:12.742871+0000","last_peered":"2024-03-26T18:01:12.742871+0000","last_clean":"2024-03-26T18:01:12.742871+0000","last_became_active":"2024-03-26T17:59:47.227336+0000","last_became_peered":"2024-03-26T17:59:47.227336+0000","last_unstale":"2024-03-26T18:01:12.742871+0000","last_undegraded":"2024-03-26T18:01:12.742871+0000","last_fullsized":"2024-03-26T18:01:12.742871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:44:25.326477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581934+0000","last_change":"2024-03-26T17:59:51.482051+0000","last_active":"2024-03-26T18:00:44.581934+0000","last_peered":"2024-03-26T18:00:44.581934+0000","last_clean":"2024-03-26T18:00:44.581934+0000","last_became_active":"2024-03-26T17:59:51.481417+0000","last_became_peered":"2024-03-26T17:59:51.481417+0000","last_unstale":"2024-03-26T18:00:44.581934+0000","last_undegraded":"2024-03-26T18:00:44.581934+0000","last_fullsized":"2024-03-26T18:00:44.581934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:41:33.294253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.594876+0000","last_change":"2024-03-26T18:00:44.594085+0000","last_active":"2024-03-26T18:00:44.594876+0000","last_peered":"2024-03-26T18:00:44.594876+0000","last_clean":"2024-03-26T18:00:44.594876+0000","last_became_active":"2024-03-26T18:00:44.585681+0000","last_became_peered":"2024-03-26T18:00:44.585681+0000","last_unstale":"2024-03-26T18:00:44.594876+0000","last_undegraded":"2024-03-26T18:00:44.594876+0000","last_fullsized":"2024-03-26T18:00:44.594876+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:35:41.421548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.756039+0000","last_change":"2024-03-26T17:59:47.241212+0000","last_active":"2024-03-26T18:01:12.756039+0000","last_peered":"2024-03-26T18:01:12.756039+0000","last_clean":"2024-03-26T18:01:12.756039+0000","last_became_active":"2024-03-26T17:59:47.240850+0000","last_became_peered":"2024-03-26T17:59:47.240850+0000","last_unstale":"2024-03-26T18:01:12.756039+0000","last_undegraded":"2024-03-26T18:01:12.756039+0000","last_fullsized":"2024-03-26T18:01:12.756039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:25:15.777349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581783+0000","last_change":"2024-03-26T17:59:51.474603+0000","last_active":"2024-03-26T18:00:44.581783+0000","last_peered":"2024-03-26T18:00:44.581783+0000","last_clean":"2024-03-26T18:00:44.581783+0000","last_became_active":"2024-03-26T17:59:51.474069+0000","last_became_peered":"2024-03-26T17:59:51.474069+0000","last_unstale":"2024-03-26T18:00:44.581783+0000","last_undegraded":"2024-03-26T18:00:44.581783+0000","last_fullsized":"2024-03-26T18:00:44.581783+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:32:05.459266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.580031+0000","last_change":"2024-03-26T17:59:46.435318+0000","last_active":"2024-03-26T18:00:44.580031+0000","last_peered":"2024-03-26T18:00:44.580031+0000","last_clean":"2024-03-26T18:00:44.580031+0000","last_became_active":"2024-03-26T17:59:46.434982+0000","last_became_peered":"2024-03-26T17:59:46.434982+0000","last_unstale":"2024-03-26T18:00:44.580031+0000","last_undegraded":"2024-03-26T18:00:44.580031+0000","last_fullsized":"2024-03-26T18:00:44.580031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:24:28.364973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.751924+0000","last_change":"2024-03-26T17:59:47.239552+0000","last_active":"2024-03-26T18:01:12.751924+0000","last_peered":"2024-03-26T18:01:12.751924+0000","last_clean":"2024-03-26T18:01:12.751924+0000","last_became_active":"2024-03-26T17:59:47.238027+0000","last_became_peered":"2024-03-26T17:59:47.238027+0000","last_unstale":"2024-03-26T18:01:12.751924+0000","last_undegraded":"2024-03-26T18:01:12.751924+0000","last_fullsized":"2024-03-26T18:01:12.751924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:49:39.158517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579810+0000","last_change":"2024-03-26T17:59:51.486887+0000","last_active":"2024-03-26T18:00:44.579810+0000","last_peered":"2024-03-26T18:00:44.579810+0000","last_clean":"2024-03-26T18:00:44.579810+0000","last_became_active":"2024-03-26T17:59:51.486426+0000","last_became_peered":"2024-03-26T17:59:51.486426+0000","last_unstale":"2024-03-26T18:00:44.579810+0000","last_undegraded":"2024-03-26T18:00:44.579810+0000","last_fullsized":"2024-03-26T18:00:44.579810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:56.561561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.592039+0000","last_change":"2024-03-26T17:59:46.437858+0000","last_active":"2024-03-26T18:00:44.592039+0000","last_peered":"2024-03-26T18:00:44.592039+0000","last_clean":"2024-03-26T18:00:44.592039+0000","last_became_active":"2024-03-26T17:59:46.437499+0000","last_became_peered":"2024-03-26T17:59:46.437499+0000","last_unstale":"2024-03-26T18:00:44.592039+0000","last_undegraded":"2024-03-26T18:00:44.592039+0000","last_fullsized":"2024-03-26T18:00:44.592039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:42:01.913000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.705983+0000","last_change":"2024-03-26T17:59:47.221959+0000","last_active":"2024-03-26T18:01:12.705983+0000","last_peered":"2024-03-26T18:01:12.705983+0000","last_clean":"2024-03-26T18:01:12.705983+0000","last_became_active":"2024-03-26T17:59:47.221550+0000","last_became_peered":"2024-03-26T17:59:47.221550+0000","last_unstale":"2024-03-26T18:01:12.705983+0000","last_undegraded":"2024-03-26T18:01:12.705983+0000","last_fullsized":"2024-03-26T18:01:12.705983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:53:21.354652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579619+0000","last_change":"2024-03-26T17:59:51.487487+0000","last_active":"2024-03-26T18:00:44.579619+0000","last_peered":"2024-03-26T18:00:44.579619+0000","last_clean":"2024-03-26T18:00:44.579619+0000","last_became_active":"2024-03-26T17:59:51.484494+0000","last_became_peered":"2024-03-26T17:59:51.484494+0000","last_unstale":"2024-03-26T18:00:44.579619+0000","last_undegraded":"2024-03-26T18:00:44.579619+0000","last_fullsized":"2024-03-26T18:00:44.579619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:29.849739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581982+0000","last_change":"2024-03-26T17:59:46.437507+0000","last_active":"2024-03-26T18:00:44.581982+0000","last_peered":"2024-03-26T18:00:44.581982+0000","last_clean":"2024-03-26T18:00:44.581982+0000","last_became_active":"2024-03-26T17:59:46.437146+0000","last_became_peered":"2024-03-26T17:59:46.437146+0000","last_unstale":"2024-03-26T18:00:44.581982+0000","last_undegraded":"2024-03-26T18:00:44.581982+0000","last_fullsized":"2024-03-26T18:00:44.581982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:03:44.924632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.748368+0000","last_change":"2024-03-26T17:59:47.227678+0000","last_active":"2024-03-26T18:01:12.748368+0000","last_peered":"2024-03-26T18:01:12.748368+0000","last_clean":"2024-03-26T18:01:12.748368+0000","last_became_active":"2024-03-26T17:59:47.224858+0000","last_became_peered":"2024-03-26T17:59:47.224858+0000","last_unstale":"2024-03-26T18:01:12.748368+0000","last_undegraded":"2024-03-26T18:01:12.748368+0000","last_fullsized":"2024-03-26T18:01:12.748368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:39.404469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579769+0000","last_change":"2024-03-26T17:59:51.487800+0000","last_active":"2024-03-26T18:00:44.579769+0000","last_peered":"2024-03-26T18:00:44.579769+0000","last_clean":"2024-03-26T18:00:44.579769+0000","last_became_active":"2024-03-26T17:59:51.486933+0000","last_became_peered":"2024-03-26T17:59:51.486933+0000","last_unstale":"2024-03-26T18:00:44.579769+0000","last_undegraded":"2024-03-26T18:00:44.579769+0000","last_fullsized":"2024-03-26T18:00:44.579769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:05:03.333414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.582009+0000","last_change":"2024-03-26T17:59:46.446331+0000","last_active":"2024-03-26T18:00:44.582009+0000","last_peered":"2024-03-26T18:00:44.582009+0000","last_clean":"2024-03-26T18:00:44.582009+0000","last_became_active":"2024-03-26T17:59:46.446013+0000","last_became_peered":"2024-03-26T17:59:46.446013+0000","last_unstale":"2024-03-26T18:00:44.582009+0000","last_undegraded":"2024-03-26T18:00:44.582009+0000","last_fullsized":"2024-03-26T18:00:44.582009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:59:40.458684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.752493+0000","last_change":"2024-03-26T17:59:47.236616+0000","last_active":"2024-03-26T18:01:12.752493+0000","last_peered":"2024-03-26T18:01:12.752493+0000","last_clean":"2024-03-26T18:01:12.752493+0000","last_became_active":"2024-03-26T17:59:47.236261+0000","last_became_peered":"2024-03-26T17:59:47.236261+0000","last_unstale":"2024-03-26T18:01:12.752493+0000","last_undegraded":"2024-03-26T18:01:12.752493+0000","last_fullsized":"2024-03-26T18:01:12.752493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:54:45.319703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579756+0000","last_change":"2024-03-26T17:59:51.482500+0000","last_active":"2024-03-26T18:00:44.579756+0000","last_peered":"2024-03-26T18:00:44.579756+0000","last_clean":"2024-03-26T18:00:44.579756+0000","last_became_active":"2024-03-26T17:59:51.482055+0000","last_became_peered":"2024-03-26T17:59:51.482055+0000","last_unstale":"2024-03-26T18:00:44.579756+0000","last_undegraded":"2024-03-26T18:00:44.579756+0000","last_fullsized":"2024-03-26T18:00:44.579756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:27:10.278683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579310+0000","last_change":"2024-03-26T17:59:46.438600+0000","last_active":"2024-03-26T18:00:44.579310+0000","last_peered":"2024-03-26T18:00:44.579310+0000","last_clean":"2024-03-26T18:00:44.579310+0000","last_became_active":"2024-03-26T17:59:46.438272+0000","last_became_peered":"2024-03-26T17:59:46.438272+0000","last_unstale":"2024-03-26T18:00:44.579310+0000","last_undegraded":"2024-03-26T18:00:44.579310+0000","last_fullsized":"2024-03-26T18:00:44.579310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:18:00.551008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.761256+0000","last_change":"2024-03-26T17:59:47.227500+0000","last_active":"2024-03-26T18:01:12.761256+0000","last_peered":"2024-03-26T18:01:12.761256+0000","last_clean":"2024-03-26T18:01:12.761256+0000","last_became_active":"2024-03-26T17:59:47.227184+0000","last_became_peered":"2024-03-26T17:59:47.227184+0000","last_unstale":"2024-03-26T18:01:12.761256+0000","last_undegraded":"2024-03-26T18:01:12.761256+0000","last_fullsized":"2024-03-26T18:01:12.761256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:41:22.876846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.592238+0000","last_change":"2024-03-26T17:59:51.479724+0000","last_active":"2024-03-26T18:00:44.592238+0000","last_peered":"2024-03-26T18:00:44.592238+0000","last_clean":"2024-03-26T18:00:44.592238+0000","last_became_active":"2024-03-26T17:59:51.479374+0000","last_became_peered":"2024-03-26T17:59:51.479374+0000","last_unstale":"2024-03-26T18:00:44.592238+0000","last_undegraded":"2024-03-26T18:00:44.592238+0000","last_fullsized":"2024-03-26T18:00:44.592238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:57:23.213546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":100,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:13.531367+0000","last_change":"2024-03-26T17:59:46.442609+0000","last_active":"2024-03-26T18:01:13.531367+0000","last_peered":"2024-03-26T18:01:13.531367+0000","last_clean":"2024-03-26T18:01:13.531367+0000","last_became_active":"2024-03-26T17:59:46.442311+0000","last_became_peered":"2024-03-26T17:59:46.442311+0000","last_unstale":"2024-03-26T18:01:13.531367+0000","last_undegraded":"2024-03-26T18:01:13.531367+0000","last_fullsized":"2024-03-26T18:01:13.531367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:02:40.141449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.761185+0000","last_change":"2024-03-26T17:59:47.228152+0000","last_active":"2024-03-26T18:01:12.761185+0000","last_peered":"2024-03-26T18:01:12.761185+0000","last_clean":"2024-03-26T18:01:12.761185+0000","last_became_active":"2024-03-26T17:59:47.227859+0000","last_became_peered":"2024-03-26T17:59:47.227859+0000","last_unstale":"2024-03-26T18:01:12.761185+0000","last_undegraded":"2024-03-26T18:01:12.761185+0000","last_fullsized":"2024-03-26T18:01:12.761185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:16:59.445039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583506+0000","last_change":"2024-03-26T17:59:51.487576+0000","last_active":"2024-03-26T18:00:44.583506+0000","last_peered":"2024-03-26T18:00:44.583506+0000","last_clean":"2024-03-26T18:00:44.583506+0000","last_became_active":"2024-03-26T17:59:51.486995+0000","last_became_peered":"2024-03-26T17:59:51.486995+0000","last_unstale":"2024-03-26T18:00:44.583506+0000","last_undegraded":"2024-03-26T18:00:44.583506+0000","last_fullsized":"2024-03-26T18:00:44.583506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:58.061260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.584625+0000","last_change":"2024-03-26T17:59:46.434433+0000","last_active":"2024-03-26T18:00:44.584625+0000","last_peered":"2024-03-26T18:00:44.584625+0000","last_clean":"2024-03-26T18:00:44.584625+0000","last_became_active":"2024-03-26T17:59:46.434117+0000","last_became_peered":"2024-03-26T17:59:46.434117+0000","last_unstale":"2024-03-26T18:00:44.584625+0000","last_undegraded":"2024-03-26T18:00:44.584625+0000","last_fullsized":"2024-03-26T18:00:44.584625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:28.083880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.756677+0000","last_change":"2024-03-26T17:59:47.232406+0000","last_active":"2024-03-26T18:01:12.756677+0000","last_peered":"2024-03-26T18:01:12.756677+0000","last_clean":"2024-03-26T18:01:12.756677+0000","last_became_active":"2024-03-26T17:59:47.232076+0000","last_became_peered":"2024-03-26T17:59:47.232076+0000","last_unstale":"2024-03-26T18:01:12.756677+0000","last_undegraded":"2024-03-26T18:01:12.756677+0000","last_fullsized":"2024-03-26T18:01:12.756677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:46:23.159106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.584933+0000","last_change":"2024-03-26T17:59:51.475555+0000","last_active":"2024-03-26T18:00:44.584933+0000","last_peered":"2024-03-26T18:00:44.584933+0000","last_clean":"2024-03-26T18:00:44.584933+0000","last_became_active":"2024-03-26T17:59:51.475136+0000","last_became_peered":"2024-03-26T17:59:51.475136+0000","last_unstale":"2024-03-26T18:00:44.584933+0000","last_undegraded":"2024-03-26T18:00:44.584933+0000","last_fullsized":"2024-03-26T18:00:44.584933+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:28:07.279237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.593495+0000","last_change":"2024-03-26T17:59:46.437510+0000","last_active":"2024-03-26T18:00:44.593495+0000","last_peered":"2024-03-26T18:00:44.593495+0000","last_clean":"2024-03-26T18:00:44.593495+0000","last_became_active":"2024-03-26T17:59:46.437191+0000","last_became_peered":"2024-03-26T17:59:46.437191+0000","last_unstale":"2024-03-26T18:00:44.593495+0000","last_undegraded":"2024-03-26T18:00:44.593495+0000","last_fullsized":"2024-03-26T18:00:44.593495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:19:13.759085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.756754+0000","last_change":"2024-03-26T17:59:47.225692+0000","last_active":"2024-03-26T18:01:12.756754+0000","last_peered":"2024-03-26T18:01:12.756754+0000","last_clean":"2024-03-26T18:01:12.756754+0000","last_became_active":"2024-03-26T17:59:47.225363+0000","last_became_peered":"2024-03-26T17:59:47.225363+0000","last_unstale":"2024-03-26T18:01:12.756754+0000","last_undegraded":"2024-03-26T18:01:12.756754+0000","last_fullsized":"2024-03-26T18:01:12.756754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:01:44.318833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743298+0000","last_change":"2024-03-26T17:59:47.227155+0000","last_active":"2024-03-26T18:01:12.743298+0000","last_peered":"2024-03-26T18:01:12.743298+0000","last_clean":"2024-03-26T18:01:12.743298+0000","last_became_active":"2024-03-26T17:59:47.226504+0000","last_became_peered":"2024-03-26T17:59:47.226504+0000","last_unstale":"2024-03-26T18:01:12.743298+0000","last_undegraded":"2024-03-26T18:01:12.743298+0000","last_fullsized":"2024-03-26T18:01:12.743298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:16:48.427819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742200+0000","last_change":"2024-03-26T17:59:47.230394+0000","last_active":"2024-03-26T18:01:12.742200+0000","last_peered":"2024-03-26T18:01:12.742200+0000","last_clean":"2024-03-26T18:01:12.742200+0000","last_became_active":"2024-03-26T17:59:47.228367+0000","last_became_peered":"2024-03-26T17:59:47.228367+0000","last_unstale":"2024-03-26T18:01:12.742200+0000","last_undegraded":"2024-03-26T18:01:12.742200+0000","last_fullsized":"2024-03-26T18:01:12.742200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:59:04.158938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.748706+0000","last_change":"2024-03-26T17:59:47.230877+0000","last_active":"2024-03-26T18:01:12.748706+0000","last_peered":"2024-03-26T18:01:12.748706+0000","last_clean":"2024-03-26T18:01:12.748706+0000","last_became_active":"2024-03-26T17:59:47.230478+0000","last_became_peered":"2024-03-26T17:59:47.230478+0000","last_unstale":"2024-03-26T18:01:12.748706+0000","last_undegraded":"2024-03-26T18:01:12.748706+0000","last_fullsized":"2024-03-26T18:01:12.748706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:46:25.867718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.751204+0000","last_change":"2024-03-26T18:00:44.593728+0000","last_active":"2024-03-26T18:01:12.751204+0000","last_peered":"2024-03-26T18:01:12.751204+0000","last_clean":"2024-03-26T18:01:12.751204+0000","last_became_active":"2024-03-26T18:00:44.593306+0000","last_became_peered":"2024-03-26T18:00:44.593306+0000","last_unstale":"2024-03-26T18:01:12.751204+0000","last_undegraded":"2024-03-26T18:01:12.751204+0000","last_fullsized":"2024-03-26T18:01:12.751204+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:06:40.142046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.750062+0000","last_change":"2024-03-26T17:59:47.227641+0000","last_active":"2024-03-26T18:01:12.750062+0000","last_peered":"2024-03-26T18:01:12.750062+0000","last_clean":"2024-03-26T18:01:12.750062+0000","last_became_active":"2024-03-26T17:59:47.226921+0000","last_became_peered":"2024-03-26T17:59:47.226921+0000","last_unstale":"2024-03-26T18:01:12.750062+0000","last_undegraded":"2024-03-26T18:01:12.750062+0000","last_fullsized":"2024-03-26T18:01:12.750062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:48.442562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743477+0000","last_change":"2024-03-26T17:59:47.232387+0000","last_active":"2024-03-26T18:01:12.743477+0000","last_peered":"2024-03-26T18:01:12.743477+0000","last_clean":"2024-03-26T18:01:12.743477+0000","last_became_active":"2024-03-26T17:59:47.231934+0000","last_became_peered":"2024-03-26T17:59:47.231934+0000","last_unstale":"2024-03-26T18:01:12.743477+0000","last_undegraded":"2024-03-26T18:01:12.743477+0000","last_fullsized":"2024-03-26T18:01:12.743477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:32.006136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.739750+0000","last_change":"2024-03-26T17:59:47.226178+0000","last_active":"2024-03-26T18:01:12.739750+0000","last_peered":"2024-03-26T18:01:12.739750+0000","last_clean":"2024-03-26T18:01:12.739750+0000","last_became_active":"2024-03-26T17:59:47.224692+0000","last_became_peered":"2024-03-26T17:59:47.224692+0000","last_unstale":"2024-03-26T18:01:12.739750+0000","last_undegraded":"2024-03-26T18:01:12.739750+0000","last_fullsized":"2024-03-26T18:01:12.739750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:18.617841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743707+0000","last_change":"2024-03-26T17:59:47.238289+0000","last_active":"2024-03-26T18:01:12.743707+0000","last_peered":"2024-03-26T18:01:12.743707+0000","last_clean":"2024-03-26T18:01:12.743707+0000","last_became_active":"2024-03-26T17:59:47.237904+0000","last_became_peered":"2024-03-26T17:59:47.237904+0000","last_unstale":"2024-03-26T18:01:12.743707+0000","last_undegraded":"2024-03-26T18:01:12.743707+0000","last_fullsized":"2024-03-26T18:01:12.743707+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:57:42.695335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.738434+0000","last_change":"2024-03-26T17:59:47.234780+0000","last_active":"2024-03-26T18:01:12.738434+0000","last_peered":"2024-03-26T18:01:12.738434+0000","last_clean":"2024-03-26T18:01:12.738434+0000","last_became_active":"2024-03-26T17:59:47.234478+0000","last_became_peered":"2024-03-26T17:59:47.234478+0000","last_unstale":"2024-03-26T18:01:12.738434+0000","last_undegraded":"2024-03-26T18:01:12.738434+0000","last_fullsized":"2024-03-26T18:01:12.738434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:57:43.675947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.746318+0000","last_change":"2024-03-26T17:59:47.240057+0000","last_active":"2024-03-26T18:01:12.746318+0000","last_peered":"2024-03-26T18:01:12.746318+0000","last_clean":"2024-03-26T18:01:12.746318+0000","last_became_active":"2024-03-26T17:59:47.239725+0000","last_became_peered":"2024-03-26T17:59:47.239725+0000","last_unstale":"2024-03-26T18:01:12.746318+0000","last_undegraded":"2024-03-26T18:01:12.746318+0000","last_fullsized":"2024-03-26T18:01:12.746318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:45.364403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.761051+0000","last_change":"2024-03-26T17:59:47.228059+0000","last_active":"2024-03-26T18:01:12.761051+0000","last_peered":"2024-03-26T18:01:12.761051+0000","last_clean":"2024-03-26T18:01:12.761051+0000","last_became_active":"2024-03-26T17:59:47.227666+0000","last_became_peered":"2024-03-26T17:59:47.227666+0000","last_unstale":"2024-03-26T18:01:12.761051+0000","last_undegraded":"2024-03-26T18:01:12.761051+0000","last_fullsized":"2024-03-26T18:01:12.761051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:19:26.440271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.748425+0000","last_change":"2024-03-26T17:59:47.228705+0000","last_active":"2024-03-26T18:01:12.748425+0000","last_peered":"2024-03-26T18:01:12.748425+0000","last_clean":"2024-03-26T18:01:12.748425+0000","last_became_active":"2024-03-26T17:59:47.228425+0000","last_became_peered":"2024-03-26T17:59:47.228425+0000","last_unstale":"2024-03-26T18:01:12.748425+0000","last_undegraded":"2024-03-26T18:01:12.748425+0000","last_fullsized":"2024-03-26T18:01:12.748425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:43:32.055816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742776+0000","last_change":"2024-03-26T17:59:47.232080+0000","last_active":"2024-03-26T18:01:12.742776+0000","last_peered":"2024-03-26T18:01:12.742776+0000","last_clean":"2024-03-26T18:01:12.742776+0000","last_became_active":"2024-03-26T17:59:47.231672+0000","last_became_peered":"2024-03-26T17:59:47.231672+0000","last_unstale":"2024-03-26T18:01:12.742776+0000","last_undegraded":"2024-03-26T18:01:12.742776+0000","last_fullsized":"2024-03-26T18:01:12.742776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:39.962065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250240,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70068,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":721,"num_read_kb":437,"num_write":252,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210204,"data_compressed":4800,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351007,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":504413,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612644,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32308,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712844,"statfs":{"total":95995035648,"available":95961952256,"internally_reserved":0,"allocated":1159168,"data_stored":1154343,"data_compressed":58496,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776169,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":508439,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":504721,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1101891,"data_compressed":56896,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":499847,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":563459,"data_compressed":22280,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":501303,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":564211,"data_compressed":22280,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":499555,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888131,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":504453,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1096941,"data_compressed":56896,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34580,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34580,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64334,"data_compressed":1600,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65086,"data_compressed":1600,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64850,"data_compressed":1600,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T18:01:20.503 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T18:01:20.569 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:20 smithi115 ceph-mon[36174]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:01:20.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:20 smithi171 ceph-mon[36033]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:01:20.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:20 smithi049 ceph-mon[32331]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:01:20.755 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:21.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:21 smithi049 ceph-mon[32331]: from='client.24998 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:01:21.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:21 smithi171 ceph-mon[36033]: from='client.24998 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:01:21.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:21 smithi115 ceph-mon[36174]: from='client.24998 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:01:22.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.437+0000 7f066d8a2700 1 -- 172.21.15.49:0/2416021429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066806ab50 msgr2=0x7f066806af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:22.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.437+0000 7f066d8a2700 1 --2- 172.21.15.49:0/2416021429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066806ab50 0x7f066806af30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f06580099c0 tx=0x7f0658009cd0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.438+0000 7f066d8a2700 1 -- 172.21.15.49:0/2416021429 shutdown_connections 2024-03-26T18:01:22.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.438+0000 7f066d8a2700 1 --2- 172.21.15.49:0/2416021429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0668107730 0x7f066810f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.438+0000 7f066d8a2700 1 --2- 172.21.15.49:0/2416021429 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066806ab50 0x7f066806af30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.438+0000 7f066d8a2700 1 --2- 172.21.15.49:0/2416021429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f066806b500 0x7f0668107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.438+0000 7f066d8a2700 1 -- 172.21.15.49:0/2416021429 >> 172.21.15.49:0/2416021429 conn(0x7f0668077300 msgr2=0x7f0668077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:22.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.438+0000 7f066d8a2700 1 -- 172.21.15.49:0/2416021429 shutdown_connections 2024-03-26T18:01:22.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.438+0000 7f066d8a2700 1 -- 172.21.15.49:0/2416021429 wait complete. 2024-03-26T18:01:22.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.438+0000 7f066d8a2700 1 Processor -- start 2024-03-26T18:01:22.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f066d8a2700 1 -- start start 2024-03-26T18:01:22.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f066d8a2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066806ab50 0x7f066819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:22.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f066d8a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066806b500 0x7f066819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:22.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f066d8a2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668107730 0x7f0668198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:22.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f066d8a2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f06681126e0 con 0x7f066806b500 2024-03-26T18:01:22.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f066d8a2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0668112560 con 0x7f066806ab50 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f066d8a2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0668112860 con 0x7f0668107730 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f0666ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066806ab50 0x7f066819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f06667fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066806b500 0x7f066819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f06677fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668107730 0x7f0668198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f0666ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066806ab50 0x7f066819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33100/0 (socket says 172.21.15.49:33100) 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f0666ffd700 1 -- 172.21.15.49:0/1575386224 learned_addr learned my addr 172.21.15.49:0/1575386224 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f06667fc700 1 -- 172.21.15.49:0/1575386224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668107730 msgr2=0x7f0668198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f06667fc700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668107730 0x7f0668198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f06667fc700 1 -- 172.21.15.49:0/1575386224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066806ab50 msgr2=0x7f066819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f06667fc700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066806ab50 0x7f066819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f06667fc700 1 -- 172.21.15.49:0/1575386224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0668199180 con 0x7f066806b500 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.439+0000 7f0666ffd700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066806ab50 0x7f066819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:22.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.440+0000 7f06667fc700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066806b500 0x7f066819ed50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0668103830 tx=0x7f065c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.440+0000 7f066c8a0700 1 -- 172.21.15.49:0/1575386224 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c004220 con 0x7f066806b500 2024-03-26T18:01:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.440+0000 7f066d8a2700 1 -- 172.21.15.49:0/1575386224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0668199470 con 0x7f066806b500 2024-03-26T18:01:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.440+0000 7f066c8a0700 1 -- 172.21.15.49:0/1575386224 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c004d10 con 0x7f066806b500 2024-03-26T18:01:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.440+0000 7f066c8a0700 1 -- 172.21.15.49:0/1575386224 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c010700 con 0x7f066806b500 2024-03-26T18:01:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.440+0000 7f066d8a2700 1 -- 172.21.15.49:0/1575386224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681a56e0 con 0x7f066806b500 2024-03-26T18:01:22.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.441+0000 7f066d8a2700 1 -- 172.21.15.49:0/1575386224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066810a900 con 0x7f066806b500 2024-03-26T18:01:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.442+0000 7f066c8a0700 1 -- 172.21.15.49:0/1575386224 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f065c004750 con 0x7f066806b500 2024-03-26T18:01:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.442+0000 7f066c8a0700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f06500a2fb0 0x7f06500a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.443+0000 7f0666ffd700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f06500a2fb0 0x7f06500a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.443+0000 7f066c8a0700 1 -- 172.21.15.49:0/1575386224 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f065c0cf4d0 con 0x7f066806b500 2024-03-26T18:01:22.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.443+0000 7f0666ffd700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f06500a2fb0 0x7f06500a5470 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0658009990 tx=0x7f0658005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:22.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.447+0000 7f066c8a0700 1 -- 172.21.15.49:0/1575386224 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f065c09bfc0 con 0x7f066806b500 2024-03-26T18:01:22.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:22 smithi049 ceph-mon[32331]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-26T18:01:22.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:22 smithi171 ceph-mon[36033]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-26T18:01:22.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.653+0000 7f066d8a2700 1 -- 172.21.15.49:0/1575386224 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0668199f00 con 0x7f06500a2fb0 2024-03-26T18:01:22.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.662+0000 7f066c8a0700 1 -- 172.21.15.49:0/1575386224 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340905 (secure 0 0 0) 0x7f0668199f00 con 0x7f06500a2fb0 2024-03-26T18:01:22.649 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 -- 172.21.15.49:0/1575386224 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f06500a2fb0 msgr2=0x7f06500a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f06500a2fb0 0x7f06500a5470 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0658009990 tx=0x7f0658005960 comp rx=0 tx=0).stop 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 -- 172.21.15.49:0/1575386224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066806b500 msgr2=0x7f066819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066806b500 0x7f066819ed50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0668103830 tx=0x7f065c00eca0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 -- 172.21.15.49:0/1575386224 shutdown_connections 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f06500a2fb0 0x7f06500a5470 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668107730 0x7f0668198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f066806ab50 0x7f066819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 --2- 172.21.15.49:0/1575386224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f066806b500 0x7f066819ed50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 -- 172.21.15.49:0/1575386224 >> 172.21.15.49:0/1575386224 conn(0x7f0668077300 msgr2=0x7f0668108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 -- 172.21.15.49:0/1575386224 shutdown_connections 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:22.666+0000 7f066d8a2700 1 -- 172.21.15.49:0/1575386224 wait complete. 2024-03-26T18:01:22.652 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-03-26T18:01:22.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:22 smithi115 ceph-mon[36174]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-26T18:01:23.404 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-03-26T18:01:20.728600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660670,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":892,"num_read_kb":576,"num_write":435,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":859,"ondisk_log_size":859,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361544,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580280,"statfs":{"total":1151940427776,"available":1151570206720,"internally_reserved":0,"allocated":12689408,"data_stored":8003576,"data_compressed":361608,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":138,"num_read_kb":73,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003655"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.755931+0000","last_change":"2024-03-26T17:59:47.232887+0000","last_active":"2024-03-26T18:01:12.755931+0000","last_peered":"2024-03-26T18:01:12.755931+0000","last_clean":"2024-03-26T18:01:12.755931+0000","last_became_active":"2024-03-26T17:59:47.232559+0000","last_became_peered":"2024-03-26T17:59:47.232559+0000","last_unstale":"2024-03-26T18:01:12.755931+0000","last_undegraded":"2024-03-26T18:01:12.755931+0000","last_fullsized":"2024-03-26T18:01:12.755931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:29:01.759592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.749635+0000","last_change":"2024-03-26T17:59:47.230719+0000","last_active":"2024-03-26T18:01:12.749635+0000","last_peered":"2024-03-26T18:01:12.749635+0000","last_clean":"2024-03-26T18:01:12.749635+0000","last_became_active":"2024-03-26T17:59:47.230411+0000","last_became_peered":"2024-03-26T17:59:47.230411+0000","last_unstale":"2024-03-26T18:01:12.749635+0000","last_undegraded":"2024-03-26T18:01:12.749635+0000","last_fullsized":"2024-03-26T18:01:12.749635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:29:21.410798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.755633+0000","last_change":"2024-03-26T17:59:47.233625+0000","last_active":"2024-03-26T18:01:12.755633+0000","last_peered":"2024-03-26T18:01:12.755633+0000","last_clean":"2024-03-26T18:01:12.755633+0000","last_became_active":"2024-03-26T17:59:47.233318+0000","last_became_peered":"2024-03-26T17:59:47.233318+0000","last_unstale":"2024-03-26T18:01:12.755633+0000","last_undegraded":"2024-03-26T18:01:12.755633+0000","last_fullsized":"2024-03-26T18:01:12.755633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:53:08.524508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742180+0000","last_change":"2024-03-26T17:59:47.234380+0000","last_active":"2024-03-26T18:01:12.742180+0000","last_peered":"2024-03-26T18:01:12.742180+0000","last_clean":"2024-03-26T18:01:12.742180+0000","last_became_active":"2024-03-26T17:59:47.233804+0000","last_became_peered":"2024-03-26T17:59:47.233804+0000","last_unstale":"2024-03-26T18:01:12.742180+0000","last_undegraded":"2024-03-26T18:01:12.742180+0000","last_fullsized":"2024-03-26T18:01:12.742180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:26:59.956704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.736646+0000","last_change":"2024-03-26T17:59:47.232523+0000","last_active":"2024-03-26T18:01:12.736646+0000","last_peered":"2024-03-26T18:01:12.736646+0000","last_clean":"2024-03-26T18:01:12.736646+0000","last_became_active":"2024-03-26T17:59:47.232183+0000","last_became_peered":"2024-03-26T17:59:47.232183+0000","last_unstale":"2024-03-26T18:01:12.736646+0000","last_undegraded":"2024-03-26T18:01:12.736646+0000","last_fullsized":"2024-03-26T18:01:12.736646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:47:06.538092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.740919+0000","last_change":"2024-03-26T17:59:47.229078+0000","last_active":"2024-03-26T18:01:12.740919+0000","last_peered":"2024-03-26T18:01:12.740919+0000","last_clean":"2024-03-26T18:01:12.740919+0000","last_became_active":"2024-03-26T17:59:47.228757+0000","last_became_peered":"2024-03-26T17:59:47.228757+0000","last_unstale":"2024-03-26T18:01:12.740919+0000","last_undegraded":"2024-03-26T18:01:12.740919+0000","last_fullsized":"2024-03-26T18:01:12.740919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:41:32.459219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.736008+0000","last_change":"2024-03-26T17:59:47.232111+0000","last_active":"2024-03-26T18:01:12.736008+0000","last_peered":"2024-03-26T18:01:12.736008+0000","last_clean":"2024-03-26T18:01:12.736008+0000","last_became_active":"2024-03-26T17:59:47.231440+0000","last_became_peered":"2024-03-26T17:59:47.231440+0000","last_unstale":"2024-03-26T18:01:12.736008+0000","last_undegraded":"2024-03-26T18:01:12.736008+0000","last_fullsized":"2024-03-26T18:01:12.736008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:23:11.607084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743981+0000","last_change":"2024-03-26T17:59:47.231875+0000","last_active":"2024-03-26T18:01:12.743981+0000","last_peered":"2024-03-26T18:01:12.743981+0000","last_clean":"2024-03-26T18:01:12.743981+0000","last_became_active":"2024-03-26T17:59:47.231534+0000","last_became_peered":"2024-03-26T17:59:47.231534+0000","last_unstale":"2024-03-26T18:01:12.743981+0000","last_undegraded":"2024-03-26T18:01:12.743981+0000","last_fullsized":"2024-03-26T18:01:12.743981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:05:33.010733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.750378+0000","last_change":"2024-03-26T17:59:47.235321+0000","last_active":"2024-03-26T18:01:12.750378+0000","last_peered":"2024-03-26T18:01:12.750378+0000","last_clean":"2024-03-26T18:01:12.750378+0000","last_became_active":"2024-03-26T17:59:47.231778+0000","last_became_peered":"2024-03-26T17:59:47.231778+0000","last_unstale":"2024-03-26T18:01:12.750378+0000","last_undegraded":"2024-03-26T18:01:12.750378+0000","last_fullsized":"2024-03-26T18:01:12.750378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:58:25.613249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.740991+0000","last_change":"2024-03-26T17:59:47.224272+0000","last_active":"2024-03-26T18:01:12.740991+0000","last_peered":"2024-03-26T18:01:12.740991+0000","last_clean":"2024-03-26T18:01:12.740991+0000","last_became_active":"2024-03-26T17:59:47.223439+0000","last_became_peered":"2024-03-26T17:59:47.223439+0000","last_unstale":"2024-03-26T18:01:12.740991+0000","last_undegraded":"2024-03-26T18:01:12.740991+0000","last_fullsized":"2024-03-26T18:01:12.740991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:28:40.428780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.758466+0000","last_change":"2024-03-26T17:59:47.227589+0000","last_active":"2024-03-26T18:01:12.758466+0000","last_peered":"2024-03-26T18:01:12.758466+0000","last_clean":"2024-03-26T18:01:12.758466+0000","last_became_active":"2024-03-26T17:59:47.227134+0000","last_became_peered":"2024-03-26T17:59:47.227134+0000","last_unstale":"2024-03-26T18:01:12.758466+0000","last_undegraded":"2024-03-26T18:01:12.758466+0000","last_fullsized":"2024-03-26T18:01:12.758466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:39:33.093899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742069+0000","last_change":"2024-03-26T17:59:47.233329+0000","last_active":"2024-03-26T18:01:12.742069+0000","last_peered":"2024-03-26T18:01:12.742069+0000","last_clean":"2024-03-26T18:01:12.742069+0000","last_became_active":"2024-03-26T17:59:47.233039+0000","last_became_peered":"2024-03-26T17:59:47.233039+0000","last_unstale":"2024-03-26T18:01:12.742069+0000","last_undegraded":"2024-03-26T18:01:12.742069+0000","last_fullsized":"2024-03-26T18:01:12.742069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:35:43.280834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.744599+0000","last_change":"2024-03-26T17:59:47.234468+0000","last_active":"2024-03-26T18:01:12.744599+0000","last_peered":"2024-03-26T18:01:12.744599+0000","last_clean":"2024-03-26T18:01:12.744599+0000","last_became_active":"2024-03-26T17:59:47.234063+0000","last_became_peered":"2024-03-26T17:59:47.234063+0000","last_unstale":"2024-03-26T18:01:12.744599+0000","last_undegraded":"2024-03-26T18:01:12.744599+0000","last_fullsized":"2024-03-26T18:01:12.744599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:25.542893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.749644+0000","last_change":"2024-03-26T17:59:47.229076+0000","last_active":"2024-03-26T18:01:12.749644+0000","last_peered":"2024-03-26T18:01:12.749644+0000","last_clean":"2024-03-26T18:01:12.749644+0000","last_became_active":"2024-03-26T17:59:47.228719+0000","last_became_peered":"2024-03-26T17:59:47.228719+0000","last_unstale":"2024-03-26T18:01:12.749644+0000","last_undegraded":"2024-03-26T18:01:12.749644+0000","last_fullsized":"2024-03-26T18:01:12.749644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:03:15.085812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.752146+0000","last_change":"2024-03-26T17:59:47.239166+0000","last_active":"2024-03-26T18:01:12.752146+0000","last_peered":"2024-03-26T18:01:12.752146+0000","last_clean":"2024-03-26T18:01:12.752146+0000","last_became_active":"2024-03-26T17:59:47.236051+0000","last_became_peered":"2024-03-26T17:59:47.236051+0000","last_unstale":"2024-03-26T18:01:12.752146+0000","last_undegraded":"2024-03-26T18:01:12.752146+0000","last_fullsized":"2024-03-26T18:01:12.752146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:46:36.724653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.760702+0000","last_change":"2024-03-26T17:59:47.231246+0000","last_active":"2024-03-26T18:01:12.760702+0000","last_peered":"2024-03-26T18:01:12.760702+0000","last_clean":"2024-03-26T18:01:12.760702+0000","last_became_active":"2024-03-26T17:59:47.228587+0000","last_became_peered":"2024-03-26T17:59:47.228587+0000","last_unstale":"2024-03-26T18:01:12.760702+0000","last_undegraded":"2024-03-26T18:01:12.760702+0000","last_fullsized":"2024-03-26T18:01:12.760702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:02:25.850361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.756052+0000","last_change":"2024-03-26T17:59:47.234790+0000","last_active":"2024-03-26T18:01:12.756052+0000","last_peered":"2024-03-26T18:01:12.756052+0000","last_clean":"2024-03-26T18:01:12.756052+0000","last_became_active":"2024-03-26T17:59:47.234436+0000","last_became_peered":"2024-03-26T17:59:47.234436+0000","last_unstale":"2024-03-26T18:01:12.756052+0000","last_undegraded":"2024-03-26T18:01:12.756052+0000","last_fullsized":"2024-03-26T18:01:12.756052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:54:26.949089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.739425+0000","last_change":"2024-03-26T17:59:47.238064+0000","last_active":"2024-03-26T18:01:12.739425+0000","last_peered":"2024-03-26T18:01:12.739425+0000","last_clean":"2024-03-26T18:01:12.739425+0000","last_became_active":"2024-03-26T17:59:47.237754+0000","last_became_peered":"2024-03-26T17:59:47.237754+0000","last_unstale":"2024-03-26T18:01:12.739425+0000","last_undegraded":"2024-03-26T18:01:12.739425+0000","last_fullsized":"2024-03-26T18:01:12.739425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:27:29.603516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.735843+0000","last_change":"2024-03-26T18:00:44.593818+0000","last_active":"2024-03-26T18:01:12.735843+0000","last_peered":"2024-03-26T18:01:12.735843+0000","last_clean":"2024-03-26T18:01:12.735843+0000","last_became_active":"2024-03-26T18:00:44.593418+0000","last_became_peered":"2024-03-26T18:00:44.593418+0000","last_unstale":"2024-03-26T18:01:12.735843+0000","last_undegraded":"2024-03-26T18:01:12.735843+0000","last_fullsized":"2024-03-26T18:01:12.735843+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:31:09.913943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581910+0000","last_change":"2024-03-26T17:59:51.480711+0000","last_active":"2024-03-26T18:00:44.581910+0000","last_peered":"2024-03-26T18:00:44.581910+0000","last_clean":"2024-03-26T18:00:44.581910+0000","last_became_active":"2024-03-26T17:59:51.480111+0000","last_became_peered":"2024-03-26T17:59:51.480111+0000","last_unstale":"2024-03-26T18:00:44.581910+0000","last_undegraded":"2024-03-26T18:00:44.581910+0000","last_fullsized":"2024-03-26T18:00:44.581910+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:40:55.456138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":111,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:17.534839+0000","last_change":"2024-03-26T17:59:46.434642+0000","last_active":"2024-03-26T18:01:17.534839+0000","last_peered":"2024-03-26T18:01:17.534839+0000","last_clean":"2024-03-26T18:01:17.534839+0000","last_became_active":"2024-03-26T17:59:46.434209+0000","last_became_peered":"2024-03-26T17:59:46.434209+0000","last_unstale":"2024-03-26T18:01:17.534839+0000","last_undegraded":"2024-03-26T18:01:17.534839+0000","last_fullsized":"2024-03-26T18:01:17.534839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:55:05.365712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743383+0000","last_change":"2024-03-26T17:59:47.239356+0000","last_active":"2024-03-26T18:01:12.743383+0000","last_peered":"2024-03-26T18:01:12.743383+0000","last_clean":"2024-03-26T18:01:12.743383+0000","last_became_active":"2024-03-26T17:59:47.239020+0000","last_became_peered":"2024-03-26T17:59:47.239020+0000","last_unstale":"2024-03-26T18:01:12.743383+0000","last_undegraded":"2024-03-26T18:01:12.743383+0000","last_fullsized":"2024-03-26T18:01:12.743383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:56.992379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579466+0000","last_change":"2024-03-26T17:59:51.486894+0000","last_active":"2024-03-26T18:00:44.579466+0000","last_peered":"2024-03-26T18:00:44.579466+0000","last_clean":"2024-03-26T18:00:44.579466+0000","last_became_active":"2024-03-26T17:59:51.486465+0000","last_became_peered":"2024-03-26T17:59:51.486465+0000","last_unstale":"2024-03-26T18:00:44.579466+0000","last_undegraded":"2024-03-26T18:00:44.579466+0000","last_fullsized":"2024-03-26T18:00:44.579466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:15:10.145415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.582869+0000","last_change":"2024-03-26T17:59:46.437908+0000","last_active":"2024-03-26T18:00:44.582869+0000","last_peered":"2024-03-26T18:00:44.582869+0000","last_clean":"2024-03-26T18:00:44.582869+0000","last_became_active":"2024-03-26T17:59:46.437592+0000","last_became_peered":"2024-03-26T17:59:46.437592+0000","last_unstale":"2024-03-26T18:00:44.582869+0000","last_undegraded":"2024-03-26T18:00:44.582869+0000","last_fullsized":"2024-03-26T18:00:44.582869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:00:05.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.755854+0000","last_change":"2024-03-26T17:59:47.240291+0000","last_active":"2024-03-26T18:01:12.755854+0000","last_peered":"2024-03-26T18:01:12.755854+0000","last_clean":"2024-03-26T18:01:12.755854+0000","last_became_active":"2024-03-26T17:59:47.239893+0000","last_became_peered":"2024-03-26T17:59:47.239893+0000","last_unstale":"2024-03-26T18:01:12.755854+0000","last_undegraded":"2024-03-26T18:01:12.755854+0000","last_fullsized":"2024-03-26T18:01:12.755854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:44:22.946199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579736+0000","last_change":"2024-03-26T17:59:51.483970+0000","last_active":"2024-03-26T18:00:44.579736+0000","last_peered":"2024-03-26T18:00:44.579736+0000","last_clean":"2024-03-26T18:00:44.579736+0000","last_became_active":"2024-03-26T17:59:51.483585+0000","last_became_peered":"2024-03-26T17:59:51.483585+0000","last_unstale":"2024-03-26T18:00:44.579736+0000","last_undegraded":"2024-03-26T18:00:44.579736+0000","last_fullsized":"2024-03-26T18:00:44.579736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:22:30.873421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.580114+0000","last_change":"2024-03-26T17:59:46.439374+0000","last_active":"2024-03-26T18:00:44.580114+0000","last_peered":"2024-03-26T18:00:44.580114+0000","last_clean":"2024-03-26T18:00:44.580114+0000","last_became_active":"2024-03-26T17:59:46.439065+0000","last_became_peered":"2024-03-26T17:59:46.439065+0000","last_unstale":"2024-03-26T18:00:44.580114+0000","last_undegraded":"2024-03-26T18:00:44.580114+0000","last_fullsized":"2024-03-26T18:00:44.580114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:45:37.954375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.755201+0000","last_change":"2024-03-26T17:59:47.237168+0000","last_active":"2024-03-26T18:01:12.755201+0000","last_peered":"2024-03-26T18:01:12.755201+0000","last_clean":"2024-03-26T18:01:12.755201+0000","last_became_active":"2024-03-26T17:59:47.232573+0000","last_became_peered":"2024-03-26T17:59:47.232573+0000","last_unstale":"2024-03-26T18:01:12.755201+0000","last_undegraded":"2024-03-26T18:01:12.755201+0000","last_fullsized":"2024-03-26T18:01:12.755201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:00:30.236249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579657+0000","last_change":"2024-03-26T17:59:51.477500+0000","last_active":"2024-03-26T18:00:44.579657+0000","last_peered":"2024-03-26T18:00:44.579657+0000","last_clean":"2024-03-26T18:00:44.579657+0000","last_became_active":"2024-03-26T17:59:51.477064+0000","last_became_peered":"2024-03-26T17:59:51.477064+0000","last_unstale":"2024-03-26T18:00:44.579657+0000","last_undegraded":"2024-03-26T18:00:44.579657+0000","last_fullsized":"2024-03-26T18:00:44.579657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:07:05.265910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579399+0000","last_change":"2024-03-26T17:59:46.437738+0000","last_active":"2024-03-26T18:00:44.579399+0000","last_peered":"2024-03-26T18:00:44.579399+0000","last_clean":"2024-03-26T18:00:44.579399+0000","last_became_active":"2024-03-26T17:59:46.437373+0000","last_became_peered":"2024-03-26T17:59:46.437373+0000","last_unstale":"2024-03-26T18:00:44.579399+0000","last_undegraded":"2024-03-26T18:00:44.579399+0000","last_fullsized":"2024-03-26T18:00:44.579399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:49:32.700437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.741852+0000","last_change":"2024-03-26T18:00:44.593603+0000","last_active":"2024-03-26T18:01:12.741852+0000","last_peered":"2024-03-26T18:01:12.741852+0000","last_clean":"2024-03-26T18:01:12.741852+0000","last_became_active":"2024-03-26T18:00:44.593218+0000","last_became_peered":"2024-03-26T18:00:44.593218+0000","last_unstale":"2024-03-26T18:01:12.741852+0000","last_undegraded":"2024-03-26T18:01:12.741852+0000","last_fullsized":"2024-03-26T18:01:12.741852+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:16:46.424715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.582262+0000","last_change":"2024-03-26T17:59:51.479332+0000","last_active":"2024-03-26T18:00:44.582262+0000","last_peered":"2024-03-26T18:00:44.582262+0000","last_clean":"2024-03-26T18:00:44.582262+0000","last_became_active":"2024-03-26T17:59:51.478857+0000","last_became_peered":"2024-03-26T17:59:51.478857+0000","last_unstale":"2024-03-26T18:00:44.582262+0000","last_undegraded":"2024-03-26T18:00:44.582262+0000","last_fullsized":"2024-03-26T18:00:44.582262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:04:30.568741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":111,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:18.521285+0000","last_change":"2024-03-26T17:59:46.438017+0000","last_active":"2024-03-26T18:01:18.521285+0000","last_peered":"2024-03-26T18:01:18.521285+0000","last_clean":"2024-03-26T18:01:18.521285+0000","last_became_active":"2024-03-26T17:59:46.437625+0000","last_became_peered":"2024-03-26T17:59:46.437625+0000","last_unstale":"2024-03-26T18:01:18.521285+0000","last_undegraded":"2024-03-26T18:01:18.521285+0000","last_fullsized":"2024-03-26T18:01:18.521285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:33:27.460626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.737565+0000","last_change":"2024-03-26T17:59:47.232764+0000","last_active":"2024-03-26T18:01:12.737565+0000","last_peered":"2024-03-26T18:01:12.737565+0000","last_clean":"2024-03-26T18:01:12.737565+0000","last_became_active":"2024-03-26T17:59:47.231672+0000","last_became_peered":"2024-03-26T17:59:47.231672+0000","last_unstale":"2024-03-26T18:01:12.737565+0000","last_undegraded":"2024-03-26T18:01:12.737565+0000","last_fullsized":"2024-03-26T18:01:12.737565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.812245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.593797+0000","last_change":"2024-03-26T18:00:44.593797+0000","last_active":"2024-03-26T18:00:44.593797+0000","last_peered":"2024-03-26T18:00:44.593797+0000","last_clean":"2024-03-26T18:00:44.593797+0000","last_became_active":"2024-03-26T18:00:44.593366+0000","last_became_peered":"2024-03-26T18:00:44.593366+0000","last_unstale":"2024-03-26T18:00:44.593797+0000","last_undegraded":"2024-03-26T18:00:44.593797+0000","last_fullsized":"2024-03-26T18:00:44.593797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:13:47.901554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.592843+0000","last_change":"2024-03-26T17:59:46.438232+0000","last_active":"2024-03-26T18:00:44.592843+0000","last_peered":"2024-03-26T18:00:44.592843+0000","last_clean":"2024-03-26T18:00:44.592843+0000","last_became_active":"2024-03-26T17:59:46.437871+0000","last_became_peered":"2024-03-26T17:59:46.437871+0000","last_unstale":"2024-03-26T18:00:44.592843+0000","last_undegraded":"2024-03-26T18:00:44.592843+0000","last_fullsized":"2024-03-26T18:00:44.592843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:08.626264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.739433+0000","last_change":"2024-03-26T17:59:47.237144+0000","last_active":"2024-03-26T18:01:12.739433+0000","last_peered":"2024-03-26T18:01:12.739433+0000","last_clean":"2024-03-26T18:01:12.739433+0000","last_became_active":"2024-03-26T17:59:47.236696+0000","last_became_peered":"2024-03-26T17:59:47.236696+0000","last_unstale":"2024-03-26T18:01:12.739433+0000","last_undegraded":"2024-03-26T18:01:12.739433+0000","last_fullsized":"2024-03-26T18:01:12.739433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:49:31.197898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583290+0000","last_change":"2024-03-26T17:59:51.490204+0000","last_active":"2024-03-26T18:00:44.583290+0000","last_peered":"2024-03-26T18:00:44.583290+0000","last_clean":"2024-03-26T18:00:44.583290+0000","last_became_active":"2024-03-26T17:59:51.489846+0000","last_became_peered":"2024-03-26T17:59:51.489846+0000","last_unstale":"2024-03-26T18:00:44.583290+0000","last_undegraded":"2024-03-26T18:00:44.583290+0000","last_fullsized":"2024-03-26T18:00:44.583290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:26:12.284436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.592836+0000","last_change":"2024-03-26T17:59:46.429699+0000","last_active":"2024-03-26T18:00:44.592836+0000","last_peered":"2024-03-26T18:00:44.592836+0000","last_clean":"2024-03-26T18:00:44.592836+0000","last_became_active":"2024-03-26T17:59:46.429319+0000","last_became_peered":"2024-03-26T17:59:46.429319+0000","last_unstale":"2024-03-26T18:00:44.592836+0000","last_undegraded":"2024-03-26T18:00:44.592836+0000","last_fullsized":"2024-03-26T18:00:44.592836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:05:15.695696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743260+0000","last_change":"2024-03-26T17:59:47.239705+0000","last_active":"2024-03-26T18:01:12.743260+0000","last_peered":"2024-03-26T18:01:12.743260+0000","last_clean":"2024-03-26T18:01:12.743260+0000","last_became_active":"2024-03-26T17:59:47.239046+0000","last_became_peered":"2024-03-26T17:59:47.239046+0000","last_unstale":"2024-03-26T18:01:12.743260+0000","last_undegraded":"2024-03-26T18:01:12.743260+0000","last_fullsized":"2024-03-26T18:01:12.743260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:08:34.930460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583964+0000","last_change":"2024-03-26T17:59:51.477511+0000","last_active":"2024-03-26T18:00:44.583964+0000","last_peered":"2024-03-26T18:00:44.583964+0000","last_clean":"2024-03-26T18:00:44.583964+0000","last_became_active":"2024-03-26T17:59:51.473949+0000","last_became_peered":"2024-03-26T17:59:51.473949+0000","last_unstale":"2024-03-26T18:00:44.583964+0000","last_undegraded":"2024-03-26T18:00:44.583964+0000","last_fullsized":"2024-03-26T18:00:44.583964+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:26:01.431977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.585193+0000","last_change":"2024-03-26T17:59:46.433235+0000","last_active":"2024-03-26T18:00:44.585193+0000","last_peered":"2024-03-26T18:00:44.585193+0000","last_clean":"2024-03-26T18:00:44.585193+0000","last_became_active":"2024-03-26T17:59:46.432929+0000","last_became_peered":"2024-03-26T17:59:46.432929+0000","last_unstale":"2024-03-26T18:00:44.585193+0000","last_undegraded":"2024-03-26T18:00:44.585193+0000","last_fullsized":"2024-03-26T18:00:44.585193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:46:23.064447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.745202+0000","last_change":"2024-03-26T17:59:47.232526+0000","last_active":"2024-03-26T18:01:12.745202+0000","last_peered":"2024-03-26T18:01:12.745202+0000","last_clean":"2024-03-26T18:01:12.745202+0000","last_became_active":"2024-03-26T17:59:47.232216+0000","last_became_peered":"2024-03-26T17:59:47.232216+0000","last_unstale":"2024-03-26T18:01:12.745202+0000","last_undegraded":"2024-03-26T18:01:12.745202+0000","last_fullsized":"2024-03-26T18:01:12.745202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:20:07.510565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581975+0000","last_change":"2024-03-26T17:59:51.482706+0000","last_active":"2024-03-26T18:00:44.581975+0000","last_peered":"2024-03-26T18:00:44.581975+0000","last_clean":"2024-03-26T18:00:44.581975+0000","last_became_active":"2024-03-26T17:59:51.482312+0000","last_became_peered":"2024-03-26T17:59:51.482312+0000","last_unstale":"2024-03-26T18:00:44.581975+0000","last_undegraded":"2024-03-26T18:00:44.581975+0000","last_fullsized":"2024-03-26T18:00:44.581975+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:42:26.614170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":176,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:19.521807+0000","last_change":"2024-03-26T17:59:46.440919+0000","last_active":"2024-03-26T18:01:19.521807+0000","last_peered":"2024-03-26T18:01:19.521807+0000","last_clean":"2024-03-26T18:01:19.521807+0000","last_became_active":"2024-03-26T17:59:46.440579+0000","last_became_peered":"2024-03-26T17:59:46.440579+0000","last_unstale":"2024-03-26T18:01:19.521807+0000","last_undegraded":"2024-03-26T18:01:19.521807+0000","last_fullsized":"2024-03-26T18:01:19.521807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:38:27.325478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.751826+0000","last_change":"2024-03-26T17:59:47.236251+0000","last_active":"2024-03-26T18:01:12.751826+0000","last_peered":"2024-03-26T18:01:12.751826+0000","last_clean":"2024-03-26T18:01:12.751826+0000","last_became_active":"2024-03-26T17:59:47.235941+0000","last_became_peered":"2024-03-26T17:59:47.235941+0000","last_unstale":"2024-03-26T18:01:12.751826+0000","last_undegraded":"2024-03-26T18:01:12.751826+0000","last_fullsized":"2024-03-26T18:01:12.751826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:50:34.821242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.578919+0000","last_change":"2024-03-26T17:59:51.480155+0000","last_active":"2024-03-26T18:00:44.578919+0000","last_peered":"2024-03-26T18:00:44.578919+0000","last_clean":"2024-03-26T18:00:44.578919+0000","last_became_active":"2024-03-26T17:59:51.479761+0000","last_became_peered":"2024-03-26T17:59:51.479761+0000","last_unstale":"2024-03-26T18:00:44.578919+0000","last_undegraded":"2024-03-26T18:00:44.578919+0000","last_fullsized":"2024-03-26T18:00:44.578919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:03:44.256586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583422+0000","last_change":"2024-03-26T17:59:46.436559+0000","last_active":"2024-03-26T18:00:44.583422+0000","last_peered":"2024-03-26T18:00:44.583422+0000","last_clean":"2024-03-26T18:00:44.583422+0000","last_became_active":"2024-03-26T17:59:46.436224+0000","last_became_peered":"2024-03-26T17:59:46.436224+0000","last_unstale":"2024-03-26T18:00:44.583422+0000","last_undegraded":"2024-03-26T18:00:44.583422+0000","last_fullsized":"2024-03-26T18:00:44.583422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:12:56.716278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.752019+0000","last_change":"2024-03-26T17:59:47.240266+0000","last_active":"2024-03-26T18:01:12.752019+0000","last_peered":"2024-03-26T18:01:12.752019+0000","last_clean":"2024-03-26T18:01:12.752019+0000","last_became_active":"2024-03-26T17:59:47.238847+0000","last_became_peered":"2024-03-26T17:59:47.238847+0000","last_unstale":"2024-03-26T18:01:12.752019+0000","last_undegraded":"2024-03-26T18:01:12.752019+0000","last_fullsized":"2024-03-26T18:01:12.752019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:47:36.325585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579951+0000","last_change":"2024-03-26T17:59:51.487207+0000","last_active":"2024-03-26T18:00:44.579951+0000","last_peered":"2024-03-26T18:00:44.579951+0000","last_clean":"2024-03-26T18:00:44.579951+0000","last_became_active":"2024-03-26T17:59:51.486351+0000","last_became_peered":"2024-03-26T17:59:51.486351+0000","last_unstale":"2024-03-26T18:00:44.579951+0000","last_undegraded":"2024-03-26T18:00:44.579951+0000","last_fullsized":"2024-03-26T18:00:44.579951+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:35:03.445677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.584497+0000","last_change":"2024-03-26T17:59:46.437839+0000","last_active":"2024-03-26T18:00:44.584497+0000","last_peered":"2024-03-26T18:00:44.584497+0000","last_clean":"2024-03-26T18:00:44.584497+0000","last_became_active":"2024-03-26T17:59:46.437515+0000","last_became_peered":"2024-03-26T17:59:46.437515+0000","last_unstale":"2024-03-26T18:00:44.584497+0000","last_undegraded":"2024-03-26T18:00:44.584497+0000","last_fullsized":"2024-03-26T18:00:44.584497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:38:34.875841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.755274+0000","last_change":"2024-03-26T17:59:47.234833+0000","last_active":"2024-03-26T18:01:12.755274+0000","last_peered":"2024-03-26T18:01:12.755274+0000","last_clean":"2024-03-26T18:01:12.755274+0000","last_became_active":"2024-03-26T17:59:47.232925+0000","last_became_peered":"2024-03-26T17:59:47.232925+0000","last_unstale":"2024-03-26T18:01:12.755274+0000","last_undegraded":"2024-03-26T18:01:12.755274+0000","last_fullsized":"2024-03-26T18:01:12.755274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:42:23.618396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579068+0000","last_change":"2024-03-26T17:59:51.481550+0000","last_active":"2024-03-26T18:00:44.579068+0000","last_peered":"2024-03-26T18:00:44.579068+0000","last_clean":"2024-03-26T18:00:44.579068+0000","last_became_active":"2024-03-26T17:59:51.481142+0000","last_became_peered":"2024-03-26T17:59:51.481142+0000","last_unstale":"2024-03-26T18:00:44.579068+0000","last_undegraded":"2024-03-26T18:00:44.579068+0000","last_fullsized":"2024-03-26T18:00:44.579068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:37.529859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'39","reported_seq":221,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:19.521412+0000","last_change":"2024-03-26T17:59:46.438936+0000","last_active":"2024-03-26T18:01:19.521412+0000","last_peered":"2024-03-26T18:01:19.521412+0000","last_clean":"2024-03-26T18:01:19.521412+0000","last_became_active":"2024-03-26T17:59:46.438609+0000","last_became_peered":"2024-03-26T17:59:46.438609+0000","last_unstale":"2024-03-26T18:01:19.521412+0000","last_undegraded":"2024-03-26T18:01:19.521412+0000","last_fullsized":"2024-03-26T18:01:19.521412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:38:36.333660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62582,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":155,"num_read_kb":139,"num_write":41,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.725607+0000","last_change":"2024-03-26T17:59:47.224495+0000","last_active":"2024-03-26T18:01:12.725607+0000","last_peered":"2024-03-26T18:01:12.725607+0000","last_clean":"2024-03-26T18:01:12.725607+0000","last_became_active":"2024-03-26T17:59:47.224184+0000","last_became_peered":"2024-03-26T17:59:47.224184+0000","last_unstale":"2024-03-26T18:01:12.725607+0000","last_undegraded":"2024-03-26T18:01:12.725607+0000","last_fullsized":"2024-03-26T18:01:12.725607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:17:23.028748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.584750+0000","last_change":"2024-03-26T17:59:51.483006+0000","last_active":"2024-03-26T18:00:44.584750+0000","last_peered":"2024-03-26T18:00:44.584750+0000","last_clean":"2024-03-26T18:00:44.584750+0000","last_became_active":"2024-03-26T17:59:51.482281+0000","last_became_peered":"2024-03-26T17:59:51.482281+0000","last_unstale":"2024-03-26T18:00:44.584750+0000","last_undegraded":"2024-03-26T18:00:44.584750+0000","last_fullsized":"2024-03-26T18:00:44.584750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:05:25.322809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583397+0000","last_change":"2024-03-26T17:59:46.440480+0000","last_active":"2024-03-26T18:00:44.583397+0000","last_peered":"2024-03-26T18:00:44.583397+0000","last_clean":"2024-03-26T18:00:44.583397+0000","last_became_active":"2024-03-26T17:59:46.439574+0000","last_became_peered":"2024-03-26T17:59:46.439574+0000","last_unstale":"2024-03-26T18:00:44.583397+0000","last_undegraded":"2024-03-26T18:00:44.583397+0000","last_fullsized":"2024-03-26T18:00:44.583397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:50:42.296440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.748020+0000","last_change":"2024-03-26T17:59:47.229944+0000","last_active":"2024-03-26T18:01:12.748020+0000","last_peered":"2024-03-26T18:01:12.748020+0000","last_clean":"2024-03-26T18:01:12.748020+0000","last_became_active":"2024-03-26T17:59:47.229626+0000","last_became_peered":"2024-03-26T17:59:47.229626+0000","last_unstale":"2024-03-26T18:01:12.748020+0000","last_undegraded":"2024-03-26T18:01:12.748020+0000","last_fullsized":"2024-03-26T18:01:12.748020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:18.998051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.596191+0000","last_change":"2024-03-26T18:00:44.596191+0000","last_active":"2024-03-26T18:00:44.596191+0000","last_peered":"2024-03-26T18:00:44.596191+0000","last_clean":"2024-03-26T18:00:44.596191+0000","last_became_active":"2024-03-26T18:00:44.595719+0000","last_became_peered":"2024-03-26T18:00:44.595719+0000","last_unstale":"2024-03-26T18:00:44.596191+0000","last_undegraded":"2024-03-26T18:00:44.596191+0000","last_fullsized":"2024-03-26T18:00:44.596191+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:50:17.967507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.585223+0000","last_change":"2024-03-26T17:59:46.435999+0000","last_active":"2024-03-26T18:00:44.585223+0000","last_peered":"2024-03-26T18:00:44.585223+0000","last_clean":"2024-03-26T18:00:44.585223+0000","last_became_active":"2024-03-26T17:59:46.435611+0000","last_became_peered":"2024-03-26T17:59:46.435611+0000","last_unstale":"2024-03-26T18:00:44.585223+0000","last_undegraded":"2024-03-26T18:00:44.585223+0000","last_fullsized":"2024-03-26T18:00:44.585223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:13:14.904792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.760591+0000","last_change":"2024-03-26T17:59:47.236395+0000","last_active":"2024-03-26T18:01:12.760591+0000","last_peered":"2024-03-26T18:01:12.760591+0000","last_clean":"2024-03-26T18:01:12.760591+0000","last_became_active":"2024-03-26T17:59:47.236055+0000","last_became_peered":"2024-03-26T17:59:47.236055+0000","last_unstale":"2024-03-26T18:01:12.760591+0000","last_undegraded":"2024-03-26T18:01:12.760591+0000","last_fullsized":"2024-03-26T18:01:12.760591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:06:26.131419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579523+0000","last_change":"2024-03-26T17:59:51.484584+0000","last_active":"2024-03-26T18:00:44.579523+0000","last_peered":"2024-03-26T18:00:44.579523+0000","last_clean":"2024-03-26T18:00:44.579523+0000","last_became_active":"2024-03-26T17:59:51.484138+0000","last_became_peered":"2024-03-26T17:59:51.484138+0000","last_unstale":"2024-03-26T18:00:44.579523+0000","last_undegraded":"2024-03-26T18:00:44.579523+0000","last_fullsized":"2024-03-26T18:00:44.579523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:49:40.163174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:15.525152+0000","last_change":"2024-03-26T17:59:46.433243+0000","last_active":"2024-03-26T18:01:15.525152+0000","last_peered":"2024-03-26T18:01:15.525152+0000","last_clean":"2024-03-26T18:01:15.525152+0000","last_became_active":"2024-03-26T17:59:46.432939+0000","last_became_peered":"2024-03-26T17:59:46.432939+0000","last_unstale":"2024-03-26T18:01:15.525152+0000","last_undegraded":"2024-03-26T18:01:15.525152+0000","last_fullsized":"2024-03-26T18:01:15.525152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:44:09.473862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.756403+0000","last_change":"2024-03-26T18:00:44.594670+0000","last_active":"2024-03-26T18:01:12.756403+0000","last_peered":"2024-03-26T18:01:12.756403+0000","last_clean":"2024-03-26T18:01:12.756403+0000","last_became_active":"2024-03-26T18:00:44.594329+0000","last_became_peered":"2024-03-26T18:00:44.594329+0000","last_unstale":"2024-03-26T18:01:12.756403+0000","last_undegraded":"2024-03-26T18:01:12.756403+0000","last_fullsized":"2024-03-26T18:01:12.756403+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:26:05.600410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.593287+0000","last_change":"2024-03-26T17:59:51.488010+0000","last_active":"2024-03-26T18:00:44.593287+0000","last_peered":"2024-03-26T18:00:44.593287+0000","last_clean":"2024-03-26T18:00:44.593287+0000","last_became_active":"2024-03-26T17:59:51.486406+0000","last_became_peered":"2024-03-26T17:59:51.486406+0000","last_unstale":"2024-03-26T18:00:44.593287+0000","last_undegraded":"2024-03-26T18:00:44.593287+0000","last_fullsized":"2024-03-26T18:00:44.593287+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:54:49.102720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579259+0000","last_change":"2024-03-26T17:59:46.438764+0000","last_active":"2024-03-26T18:00:44.579259+0000","last_peered":"2024-03-26T18:00:44.579259+0000","last_clean":"2024-03-26T18:00:44.579259+0000","last_became_active":"2024-03-26T17:59:46.438329+0000","last_became_peered":"2024-03-26T17:59:46.438329+0000","last_unstale":"2024-03-26T18:00:44.579259+0000","last_undegraded":"2024-03-26T18:00:44.579259+0000","last_fullsized":"2024-03-26T18:00:44.579259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:33:19.025134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.738879+0000","last_change":"2024-03-26T17:59:47.237020+0000","last_active":"2024-03-26T18:01:12.738879+0000","last_peered":"2024-03-26T18:01:12.738879+0000","last_clean":"2024-03-26T18:01:12.738879+0000","last_became_active":"2024-03-26T17:59:47.236693+0000","last_became_peered":"2024-03-26T17:59:47.236693+0000","last_unstale":"2024-03-26T18:01:12.738879+0000","last_undegraded":"2024-03-26T18:01:12.738879+0000","last_fullsized":"2024-03-26T18:01:12.738879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:40:25.473994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.584148+0000","last_change":"2024-03-26T17:59:51.482799+0000","last_active":"2024-03-26T18:00:44.584148+0000","last_peered":"2024-03-26T18:00:44.584148+0000","last_clean":"2024-03-26T18:00:44.584148+0000","last_became_active":"2024-03-26T17:59:51.482246+0000","last_became_peered":"2024-03-26T17:59:51.482246+0000","last_unstale":"2024-03-26T18:00:44.584148+0000","last_undegraded":"2024-03-26T18:00:44.584148+0000","last_fullsized":"2024-03-26T18:00:44.584148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:43:11.132952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579701+0000","last_change":"2024-03-26T17:59:46.438038+0000","last_active":"2024-03-26T18:00:44.579701+0000","last_peered":"2024-03-26T18:00:44.579701+0000","last_clean":"2024-03-26T18:00:44.579701+0000","last_became_active":"2024-03-26T17:59:46.437654+0000","last_became_peered":"2024-03-26T17:59:46.437654+0000","last_unstale":"2024-03-26T18:00:44.579701+0000","last_undegraded":"2024-03-26T18:00:44.579701+0000","last_fullsized":"2024-03-26T18:00:44.579701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:33:04.341028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.738114+0000","last_change":"2024-03-26T18:00:44.596184+0000","last_active":"2024-03-26T18:01:12.738114+0000","last_peered":"2024-03-26T18:01:12.738114+0000","last_clean":"2024-03-26T18:01:12.738114+0000","last_became_active":"2024-03-26T18:00:44.595766+0000","last_became_peered":"2024-03-26T18:00:44.595766+0000","last_unstale":"2024-03-26T18:01:12.738114+0000","last_undegraded":"2024-03-26T18:01:12.738114+0000","last_fullsized":"2024-03-26T18:01:12.738114+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:02:33.148661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579514+0000","last_change":"2024-03-26T17:59:51.477057+0000","last_active":"2024-03-26T18:00:44.579514+0000","last_peered":"2024-03-26T18:00:44.579514+0000","last_clean":"2024-03-26T18:00:44.579514+0000","last_became_active":"2024-03-26T17:59:51.476576+0000","last_became_peered":"2024-03-26T17:59:51.476576+0000","last_unstale":"2024-03-26T18:00:44.579514+0000","last_undegraded":"2024-03-26T18:00:44.579514+0000","last_fullsized":"2024-03-26T18:00:44.579514+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:42.533519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.587240+0000","last_change":"2024-03-26T18:00:44.587240+0000","last_active":"2024-03-26T18:00:44.587240+0000","last_peered":"2024-03-26T18:00:44.587240+0000","last_clean":"2024-03-26T18:00:44.587240+0000","last_became_active":"2024-03-26T18:00:44.586857+0000","last_became_peered":"2024-03-26T18:00:44.586857+0000","last_unstale":"2024-03-26T18:00:44.587240+0000","last_undegraded":"2024-03-26T18:00:44.587240+0000","last_fullsized":"2024-03-26T18:00:44.587240+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:28:02.590070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.738117+0000","last_change":"2024-03-26T17:59:47.232436+0000","last_active":"2024-03-26T18:01:12.738117+0000","last_peered":"2024-03-26T18:01:12.738117+0000","last_clean":"2024-03-26T18:01:12.738117+0000","last_became_active":"2024-03-26T17:59:47.232129+0000","last_became_peered":"2024-03-26T17:59:47.232129+0000","last_unstale":"2024-03-26T18:01:12.738117+0000","last_undegraded":"2024-03-26T18:01:12.738117+0000","last_fullsized":"2024-03-26T18:01:12.738117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:27:54.504624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.593148+0000","last_change":"2024-03-26T17:59:51.475972+0000","last_active":"2024-03-26T18:00:44.593148+0000","last_peered":"2024-03-26T18:00:44.593148+0000","last_clean":"2024-03-26T18:00:44.593148+0000","last_became_active":"2024-03-26T17:59:51.475531+0000","last_became_peered":"2024-03-26T17:59:51.475531+0000","last_unstale":"2024-03-26T18:00:44.593148+0000","last_undegraded":"2024-03-26T18:00:44.593148+0000","last_fullsized":"2024-03-26T18:00:44.593148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:43:23.639809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:16.534095+0000","last_change":"2024-03-26T17:59:46.432980+0000","last_active":"2024-03-26T18:01:16.534095+0000","last_peered":"2024-03-26T18:01:16.534095+0000","last_clean":"2024-03-26T18:01:16.534095+0000","last_became_active":"2024-03-26T17:59:46.432095+0000","last_became_peered":"2024-03-26T17:59:46.432095+0000","last_unstale":"2024-03-26T18:01:16.534095+0000","last_undegraded":"2024-03-26T18:01:16.534095+0000","last_fullsized":"2024-03-26T18:01:16.534095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:19:36.897555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.747649+0000","last_change":"2024-03-26T17:59:47.229623+0000","last_active":"2024-03-26T18:01:12.747649+0000","last_peered":"2024-03-26T18:01:12.747649+0000","last_clean":"2024-03-26T18:01:12.747649+0000","last_became_active":"2024-03-26T17:59:47.228781+0000","last_became_peered":"2024-03-26T17:59:47.228781+0000","last_unstale":"2024-03-26T18:01:12.747649+0000","last_undegraded":"2024-03-26T18:01:12.747649+0000","last_fullsized":"2024-03-26T18:01:12.747649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:54:22.171273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.593230+0000","last_change":"2024-03-26T17:59:51.483247+0000","last_active":"2024-03-26T18:00:44.593230+0000","last_peered":"2024-03-26T18:00:44.593230+0000","last_clean":"2024-03-26T18:00:44.593230+0000","last_became_active":"2024-03-26T17:59:51.482854+0000","last_became_peered":"2024-03-26T17:59:51.482854+0000","last_unstale":"2024-03-26T18:00:44.593230+0000","last_undegraded":"2024-03-26T18:00:44.593230+0000","last_fullsized":"2024-03-26T18:00:44.593230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:24:13.532986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.733522+0000","last_change":"2024-03-26T17:59:47.238866+0000","last_active":"2024-03-26T18:01:12.733522+0000","last_peered":"2024-03-26T18:01:12.733522+0000","last_clean":"2024-03-26T18:01:12.733522+0000","last_became_active":"2024-03-26T17:59:47.238366+0000","last_became_peered":"2024-03-26T17:59:47.238366+0000","last_unstale":"2024-03-26T18:01:12.733522+0000","last_undegraded":"2024-03-26T18:01:12.733522+0000","last_fullsized":"2024-03-26T18:01:12.733522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:04.045820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:15.524784+0000","last_change":"2024-03-26T17:59:46.433046+0000","last_active":"2024-03-26T18:01:15.524784+0000","last_peered":"2024-03-26T18:01:15.524784+0000","last_clean":"2024-03-26T18:01:15.524784+0000","last_became_active":"2024-03-26T17:59:46.432585+0000","last_became_peered":"2024-03-26T17:59:46.432585+0000","last_unstale":"2024-03-26T18:01:15.524784+0000","last_undegraded":"2024-03-26T18:01:15.524784+0000","last_fullsized":"2024-03-26T18:01:15.524784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:17.051521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583509+0000","last_change":"2024-03-26T17:59:51.478866+0000","last_active":"2024-03-26T18:00:44.583509+0000","last_peered":"2024-03-26T18:00:44.583509+0000","last_clean":"2024-03-26T18:00:44.583509+0000","last_became_active":"2024-03-26T17:59:51.478497+0000","last_became_peered":"2024-03-26T17:59:51.478497+0000","last_unstale":"2024-03-26T18:00:44.583509+0000","last_undegraded":"2024-03-26T18:00:44.583509+0000","last_fullsized":"2024-03-26T18:00:44.583509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:15:05.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.747008+0000","last_change":"2024-03-26T17:59:47.232349+0000","last_active":"2024-03-26T18:01:12.747008+0000","last_peered":"2024-03-26T18:01:12.747008+0000","last_clean":"2024-03-26T18:01:12.747008+0000","last_became_active":"2024-03-26T17:59:47.232001+0000","last_became_peered":"2024-03-26T17:59:47.232001+0000","last_unstale":"2024-03-26T18:01:12.747008+0000","last_undegraded":"2024-03-26T18:01:12.747008+0000","last_fullsized":"2024-03-26T18:01:12.747008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:42:14.063248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579850+0000","last_change":"2024-03-26T17:59:46.431522+0000","last_active":"2024-03-26T18:00:44.579850+0000","last_peered":"2024-03-26T18:00:44.579850+0000","last_clean":"2024-03-26T18:00:44.579850+0000","last_became_active":"2024-03-26T17:59:46.431201+0000","last_became_peered":"2024-03-26T17:59:46.431201+0000","last_unstale":"2024-03-26T18:00:44.579850+0000","last_undegraded":"2024-03-26T18:00:44.579850+0000","last_fullsized":"2024-03-26T18:00:44.579850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:11.298111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.582849+0000","last_change":"2024-03-26T17:58:38.167621+0000","last_active":"2024-03-26T18:00:44.582849+0000","last_peered":"2024-03-26T18:00:44.582849+0000","last_clean":"2024-03-26T18:00:44.582849+0000","last_became_active":"2024-03-26T17:58:38.167247+0000","last_became_peered":"2024-03-26T17:58:38.167247+0000","last_unstale":"2024-03-26T18:00:44.582849+0000","last_undegraded":"2024-03-26T18:00:44.582849+0000","last_fullsized":"2024-03-26T18:00:44.582849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:55:11.216723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:55:11.216723+0000","last_clean_scrub_stamp":"2024-03-26T17:55:11.216723+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:59:04.140225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.585109+0000","last_change":"2024-03-26T17:59:51.480354+0000","last_active":"2024-03-26T18:00:44.585109+0000","last_peered":"2024-03-26T18:00:44.585109+0000","last_clean":"2024-03-26T18:00:44.585109+0000","last_became_active":"2024-03-26T17:59:51.479474+0000","last_became_peered":"2024-03-26T17:59:51.479474+0000","last_unstale":"2024-03-26T18:00:44.585109+0000","last_undegraded":"2024-03-26T18:00:44.585109+0000","last_fullsized":"2024-03-26T18:00:44.585109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:48:31.846364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742931+0000","last_change":"2024-03-26T17:59:47.226036+0000","last_active":"2024-03-26T18:01:12.742931+0000","last_peered":"2024-03-26T18:01:12.742931+0000","last_clean":"2024-03-26T18:01:12.742931+0000","last_became_active":"2024-03-26T17:59:47.225722+0000","last_became_peered":"2024-03-26T17:59:47.225722+0000","last_unstale":"2024-03-26T18:01:12.742931+0000","last_undegraded":"2024-03-26T18:01:12.742931+0000","last_fullsized":"2024-03-26T18:01:12.742931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:28:39.310849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.582900+0000","last_change":"2024-03-26T17:59:46.432635+0000","last_active":"2024-03-26T18:00:44.582900+0000","last_peered":"2024-03-26T18:00:44.582900+0000","last_clean":"2024-03-26T18:00:44.582900+0000","last_became_active":"2024-03-26T17:59:46.432317+0000","last_became_peered":"2024-03-26T17:59:46.432317+0000","last_unstale":"2024-03-26T18:00:44.582900+0000","last_undegraded":"2024-03-26T18:00:44.582900+0000","last_fullsized":"2024-03-26T18:00:44.582900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:48:15.018906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581902+0000","last_change":"2024-03-26T17:59:51.480678+0000","last_active":"2024-03-26T18:00:44.581902+0000","last_peered":"2024-03-26T18:00:44.581902+0000","last_clean":"2024-03-26T18:00:44.581902+0000","last_became_active":"2024-03-26T17:59:51.480290+0000","last_became_peered":"2024-03-26T17:59:51.480290+0000","last_unstale":"2024-03-26T18:00:44.581902+0000","last_undegraded":"2024-03-26T18:00:44.581902+0000","last_fullsized":"2024-03-26T18:00:44.581902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:02:20.279963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.580026+0000","last_change":"2024-03-26T17:59:46.445569+0000","last_active":"2024-03-26T18:00:44.580026+0000","last_peered":"2024-03-26T18:00:44.580026+0000","last_clean":"2024-03-26T18:00:44.580026+0000","last_became_active":"2024-03-26T17:59:46.445114+0000","last_became_peered":"2024-03-26T17:59:46.445114+0000","last_unstale":"2024-03-26T18:00:44.580026+0000","last_undegraded":"2024-03-26T18:00:44.580026+0000","last_fullsized":"2024-03-26T18:00:44.580026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:18:59.367401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742871+0000","last_change":"2024-03-26T17:59:47.227901+0000","last_active":"2024-03-26T18:01:12.742871+0000","last_peered":"2024-03-26T18:01:12.742871+0000","last_clean":"2024-03-26T18:01:12.742871+0000","last_became_active":"2024-03-26T17:59:47.227336+0000","last_became_peered":"2024-03-26T17:59:47.227336+0000","last_unstale":"2024-03-26T18:01:12.742871+0000","last_undegraded":"2024-03-26T18:01:12.742871+0000","last_fullsized":"2024-03-26T18:01:12.742871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:44:25.326477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581934+0000","last_change":"2024-03-26T17:59:51.482051+0000","last_active":"2024-03-26T18:00:44.581934+0000","last_peered":"2024-03-26T18:00:44.581934+0000","last_clean":"2024-03-26T18:00:44.581934+0000","last_became_active":"2024-03-26T17:59:51.481417+0000","last_became_peered":"2024-03-26T17:59:51.481417+0000","last_unstale":"2024-03-26T18:00:44.581934+0000","last_undegraded":"2024-03-26T18:00:44.581934+0000","last_fullsized":"2024-03-26T18:00:44.581934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:41:33.294253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.594876+0000","last_change":"2024-03-26T18:00:44.594085+0000","last_active":"2024-03-26T18:00:44.594876+0000","last_peered":"2024-03-26T18:00:44.594876+0000","last_clean":"2024-03-26T18:00:44.594876+0000","last_became_active":"2024-03-26T18:00:44.585681+0000","last_became_peered":"2024-03-26T18:00:44.585681+0000","last_unstale":"2024-03-26T18:00:44.594876+0000","last_undegraded":"2024-03-26T18:00:44.594876+0000","last_fullsized":"2024-03-26T18:00:44.594876+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:35:41.421548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.756039+0000","last_change":"2024-03-26T17:59:47.241212+0000","last_active":"2024-03-26T18:01:12.756039+0000","last_peered":"2024-03-26T18:01:12.756039+0000","last_clean":"2024-03-26T18:01:12.756039+0000","last_became_active":"2024-03-26T17:59:47.240850+0000","last_became_peered":"2024-03-26T17:59:47.240850+0000","last_unstale":"2024-03-26T18:01:12.756039+0000","last_undegraded":"2024-03-26T18:01:12.756039+0000","last_fullsized":"2024-03-26T18:01:12.756039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:25:15.777349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581783+0000","last_change":"2024-03-26T17:59:51.474603+0000","last_active":"2024-03-26T18:00:44.581783+0000","last_peered":"2024-03-26T18:00:44.581783+0000","last_clean":"2024-03-26T18:00:44.581783+0000","last_became_active":"2024-03-26T17:59:51.474069+0000","last_became_peered":"2024-03-26T17:59:51.474069+0000","last_unstale":"2024-03-26T18:00:44.581783+0000","last_undegraded":"2024-03-26T18:00:44.581783+0000","last_fullsized":"2024-03-26T18:00:44.581783+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:32:05.459266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.580031+0000","last_change":"2024-03-26T17:59:46.435318+0000","last_active":"2024-03-26T18:00:44.580031+0000","last_peered":"2024-03-26T18:00:44.580031+0000","last_clean":"2024-03-26T18:00:44.580031+0000","last_became_active":"2024-03-26T17:59:46.434982+0000","last_became_peered":"2024-03-26T17:59:46.434982+0000","last_unstale":"2024-03-26T18:00:44.580031+0000","last_undegraded":"2024-03-26T18:00:44.580031+0000","last_fullsized":"2024-03-26T18:00:44.580031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:24:28.364973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.751924+0000","last_change":"2024-03-26T17:59:47.239552+0000","last_active":"2024-03-26T18:01:12.751924+0000","last_peered":"2024-03-26T18:01:12.751924+0000","last_clean":"2024-03-26T18:01:12.751924+0000","last_became_active":"2024-03-26T17:59:47.238027+0000","last_became_peered":"2024-03-26T17:59:47.238027+0000","last_unstale":"2024-03-26T18:01:12.751924+0000","last_undegraded":"2024-03-26T18:01:12.751924+0000","last_fullsized":"2024-03-26T18:01:12.751924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:49:39.158517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579810+0000","last_change":"2024-03-26T17:59:51.486887+0000","last_active":"2024-03-26T18:00:44.579810+0000","last_peered":"2024-03-26T18:00:44.579810+0000","last_clean":"2024-03-26T18:00:44.579810+0000","last_became_active":"2024-03-26T17:59:51.486426+0000","last_became_peered":"2024-03-26T17:59:51.486426+0000","last_unstale":"2024-03-26T18:00:44.579810+0000","last_undegraded":"2024-03-26T18:00:44.579810+0000","last_fullsized":"2024-03-26T18:00:44.579810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:56.561561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.592039+0000","last_change":"2024-03-26T17:59:46.437858+0000","last_active":"2024-03-26T18:00:44.592039+0000","last_peered":"2024-03-26T18:00:44.592039+0000","last_clean":"2024-03-26T18:00:44.592039+0000","last_became_active":"2024-03-26T17:59:46.437499+0000","last_became_peered":"2024-03-26T17:59:46.437499+0000","last_unstale":"2024-03-26T18:00:44.592039+0000","last_undegraded":"2024-03-26T18:00:44.592039+0000","last_fullsized":"2024-03-26T18:00:44.592039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:42:01.913000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.705983+0000","last_change":"2024-03-26T17:59:47.221959+0000","last_active":"2024-03-26T18:01:12.705983+0000","last_peered":"2024-03-26T18:01:12.705983+0000","last_clean":"2024-03-26T18:01:12.705983+0000","last_became_active":"2024-03-26T17:59:47.221550+0000","last_became_peered":"2024-03-26T17:59:47.221550+0000","last_unstale":"2024-03-26T18:01:12.705983+0000","last_undegraded":"2024-03-26T18:01:12.705983+0000","last_fullsized":"2024-03-26T18:01:12.705983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:53:21.354652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579619+0000","last_change":"2024-03-26T17:59:51.487487+0000","last_active":"2024-03-26T18:00:44.579619+0000","last_peered":"2024-03-26T18:00:44.579619+0000","last_clean":"2024-03-26T18:00:44.579619+0000","last_became_active":"2024-03-26T17:59:51.484494+0000","last_became_peered":"2024-03-26T17:59:51.484494+0000","last_unstale":"2024-03-26T18:00:44.579619+0000","last_undegraded":"2024-03-26T18:00:44.579619+0000","last_fullsized":"2024-03-26T18:00:44.579619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:29.849739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.581982+0000","last_change":"2024-03-26T17:59:46.437507+0000","last_active":"2024-03-26T18:00:44.581982+0000","last_peered":"2024-03-26T18:00:44.581982+0000","last_clean":"2024-03-26T18:00:44.581982+0000","last_became_active":"2024-03-26T17:59:46.437146+0000","last_became_peered":"2024-03-26T17:59:46.437146+0000","last_unstale":"2024-03-26T18:00:44.581982+0000","last_undegraded":"2024-03-26T18:00:44.581982+0000","last_fullsized":"2024-03-26T18:00:44.581982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:03:44.924632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.748368+0000","last_change":"2024-03-26T17:59:47.227678+0000","last_active":"2024-03-26T18:01:12.748368+0000","last_peered":"2024-03-26T18:01:12.748368+0000","last_clean":"2024-03-26T18:01:12.748368+0000","last_became_active":"2024-03-26T17:59:47.224858+0000","last_became_peered":"2024-03-26T17:59:47.224858+0000","last_unstale":"2024-03-26T18:01:12.748368+0000","last_undegraded":"2024-03-26T18:01:12.748368+0000","last_fullsized":"2024-03-26T18:01:12.748368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:39.404469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579769+0000","last_change":"2024-03-26T17:59:51.487800+0000","last_active":"2024-03-26T18:00:44.579769+0000","last_peered":"2024-03-26T18:00:44.579769+0000","last_clean":"2024-03-26T18:00:44.579769+0000","last_became_active":"2024-03-26T17:59:51.486933+0000","last_became_peered":"2024-03-26T17:59:51.486933+0000","last_unstale":"2024-03-26T18:00:44.579769+0000","last_undegraded":"2024-03-26T18:00:44.579769+0000","last_fullsized":"2024-03-26T18:00:44.579769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:05:03.333414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.582009+0000","last_change":"2024-03-26T17:59:46.446331+0000","last_active":"2024-03-26T18:00:44.582009+0000","last_peered":"2024-03-26T18:00:44.582009+0000","last_clean":"2024-03-26T18:00:44.582009+0000","last_became_active":"2024-03-26T17:59:46.446013+0000","last_became_peered":"2024-03-26T17:59:46.446013+0000","last_unstale":"2024-03-26T18:00:44.582009+0000","last_undegraded":"2024-03-26T18:00:44.582009+0000","last_fullsized":"2024-03-26T18:00:44.582009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:59:40.458684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.752493+0000","last_change":"2024-03-26T17:59:47.236616+0000","last_active":"2024-03-26T18:01:12.752493+0000","last_peered":"2024-03-26T18:01:12.752493+0000","last_clean":"2024-03-26T18:01:12.752493+0000","last_became_active":"2024-03-26T17:59:47.236261+0000","last_became_peered":"2024-03-26T17:59:47.236261+0000","last_unstale":"2024-03-26T18:01:12.752493+0000","last_undegraded":"2024-03-26T18:01:12.752493+0000","last_fullsized":"2024-03-26T18:01:12.752493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:54:45.319703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579756+0000","last_change":"2024-03-26T17:59:51.482500+0000","last_active":"2024-03-26T18:00:44.579756+0000","last_peered":"2024-03-26T18:00:44.579756+0000","last_clean":"2024-03-26T18:00:44.579756+0000","last_became_active":"2024-03-26T17:59:51.482055+0000","last_became_peered":"2024-03-26T17:59:51.482055+0000","last_unstale":"2024-03-26T18:00:44.579756+0000","last_undegraded":"2024-03-26T18:00:44.579756+0000","last_fullsized":"2024-03-26T18:00:44.579756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:27:10.278683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.579310+0000","last_change":"2024-03-26T17:59:46.438600+0000","last_active":"2024-03-26T18:00:44.579310+0000","last_peered":"2024-03-26T18:00:44.579310+0000","last_clean":"2024-03-26T18:00:44.579310+0000","last_became_active":"2024-03-26T17:59:46.438272+0000","last_became_peered":"2024-03-26T17:59:46.438272+0000","last_unstale":"2024-03-26T18:00:44.579310+0000","last_undegraded":"2024-03-26T18:00:44.579310+0000","last_fullsized":"2024-03-26T18:00:44.579310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:18:00.551008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.761256+0000","last_change":"2024-03-26T17:59:47.227500+0000","last_active":"2024-03-26T18:01:12.761256+0000","last_peered":"2024-03-26T18:01:12.761256+0000","last_clean":"2024-03-26T18:01:12.761256+0000","last_became_active":"2024-03-26T17:59:47.227184+0000","last_became_peered":"2024-03-26T17:59:47.227184+0000","last_unstale":"2024-03-26T18:01:12.761256+0000","last_undegraded":"2024-03-26T18:01:12.761256+0000","last_fullsized":"2024-03-26T18:01:12.761256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:41:22.876846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.592238+0000","last_change":"2024-03-26T17:59:51.479724+0000","last_active":"2024-03-26T18:00:44.592238+0000","last_peered":"2024-03-26T18:00:44.592238+0000","last_clean":"2024-03-26T18:00:44.592238+0000","last_became_active":"2024-03-26T17:59:51.479374+0000","last_became_peered":"2024-03-26T17:59:51.479374+0000","last_unstale":"2024-03-26T18:00:44.592238+0000","last_undegraded":"2024-03-26T18:00:44.592238+0000","last_fullsized":"2024-03-26T18:00:44.592238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:57:23.213546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:18.536433+0000","last_change":"2024-03-26T17:59:46.442609+0000","last_active":"2024-03-26T18:01:18.536433+0000","last_peered":"2024-03-26T18:01:18.536433+0000","last_clean":"2024-03-26T18:01:18.536433+0000","last_became_active":"2024-03-26T17:59:46.442311+0000","last_became_peered":"2024-03-26T17:59:46.442311+0000","last_unstale":"2024-03-26T18:01:18.536433+0000","last_undegraded":"2024-03-26T18:01:18.536433+0000","last_fullsized":"2024-03-26T18:01:18.536433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:02:40.141449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.761185+0000","last_change":"2024-03-26T17:59:47.228152+0000","last_active":"2024-03-26T18:01:12.761185+0000","last_peered":"2024-03-26T18:01:12.761185+0000","last_clean":"2024-03-26T18:01:12.761185+0000","last_became_active":"2024-03-26T17:59:47.227859+0000","last_became_peered":"2024-03-26T17:59:47.227859+0000","last_unstale":"2024-03-26T18:01:12.761185+0000","last_undegraded":"2024-03-26T18:01:12.761185+0000","last_fullsized":"2024-03-26T18:01:12.761185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:16:59.445039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.583506+0000","last_change":"2024-03-26T17:59:51.487576+0000","last_active":"2024-03-26T18:00:44.583506+0000","last_peered":"2024-03-26T18:00:44.583506+0000","last_clean":"2024-03-26T18:00:44.583506+0000","last_became_active":"2024-03-26T17:59:51.486995+0000","last_became_peered":"2024-03-26T17:59:51.486995+0000","last_unstale":"2024-03-26T18:00:44.583506+0000","last_undegraded":"2024-03-26T18:00:44.583506+0000","last_fullsized":"2024-03-26T18:00:44.583506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:58.061260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.584625+0000","last_change":"2024-03-26T17:59:46.434433+0000","last_active":"2024-03-26T18:00:44.584625+0000","last_peered":"2024-03-26T18:00:44.584625+0000","last_clean":"2024-03-26T18:00:44.584625+0000","last_became_active":"2024-03-26T17:59:46.434117+0000","last_became_peered":"2024-03-26T17:59:46.434117+0000","last_unstale":"2024-03-26T18:00:44.584625+0000","last_undegraded":"2024-03-26T18:00:44.584625+0000","last_fullsized":"2024-03-26T18:00:44.584625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:28.083880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.756677+0000","last_change":"2024-03-26T17:59:47.232406+0000","last_active":"2024-03-26T18:01:12.756677+0000","last_peered":"2024-03-26T18:01:12.756677+0000","last_clean":"2024-03-26T18:01:12.756677+0000","last_became_active":"2024-03-26T17:59:47.232076+0000","last_became_peered":"2024-03-26T17:59:47.232076+0000","last_unstale":"2024-03-26T18:01:12.756677+0000","last_undegraded":"2024-03-26T18:01:12.756677+0000","last_fullsized":"2024-03-26T18:01:12.756677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:46:23.159106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.584933+0000","last_change":"2024-03-26T17:59:51.475555+0000","last_active":"2024-03-26T18:00:44.584933+0000","last_peered":"2024-03-26T18:00:44.584933+0000","last_clean":"2024-03-26T18:00:44.584933+0000","last_became_active":"2024-03-26T17:59:51.475136+0000","last_became_peered":"2024-03-26T17:59:51.475136+0000","last_unstale":"2024-03-26T18:00:44.584933+0000","last_undegraded":"2024-03-26T18:00:44.584933+0000","last_fullsized":"2024-03-26T18:00:44.584933+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:50.450629+0000","last_clean_scrub_stamp":"2024-03-26T17:59:50.450629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:28:07.279237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:00:44.593495+0000","last_change":"2024-03-26T17:59:46.437510+0000","last_active":"2024-03-26T18:00:44.593495+0000","last_peered":"2024-03-26T18:00:44.593495+0000","last_clean":"2024-03-26T18:00:44.593495+0000","last_became_active":"2024-03-26T17:59:46.437191+0000","last_became_peered":"2024-03-26T17:59:46.437191+0000","last_unstale":"2024-03-26T18:00:44.593495+0000","last_undegraded":"2024-03-26T18:00:44.593495+0000","last_fullsized":"2024-03-26T18:00:44.593495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:45.406409+0000","last_clean_scrub_stamp":"2024-03-26T17:59:45.406409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:19:13.759085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.756754+0000","last_change":"2024-03-26T17:59:47.225692+0000","last_active":"2024-03-26T18:01:12.756754+0000","last_peered":"2024-03-26T18:01:12.756754+0000","last_clean":"2024-03-26T18:01:12.756754+0000","last_became_active":"2024-03-26T17:59:47.225363+0000","last_became_peered":"2024-03-26T17:59:47.225363+0000","last_unstale":"2024-03-26T18:01:12.756754+0000","last_undegraded":"2024-03-26T18:01:12.756754+0000","last_fullsized":"2024-03-26T18:01:12.756754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:01:44.318833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743298+0000","last_change":"2024-03-26T17:59:47.227155+0000","last_active":"2024-03-26T18:01:12.743298+0000","last_peered":"2024-03-26T18:01:12.743298+0000","last_clean":"2024-03-26T18:01:12.743298+0000","last_became_active":"2024-03-26T17:59:47.226504+0000","last_became_peered":"2024-03-26T17:59:47.226504+0000","last_unstale":"2024-03-26T18:01:12.743298+0000","last_undegraded":"2024-03-26T18:01:12.743298+0000","last_fullsized":"2024-03-26T18:01:12.743298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:16:48.427819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742200+0000","last_change":"2024-03-26T17:59:47.230394+0000","last_active":"2024-03-26T18:01:12.742200+0000","last_peered":"2024-03-26T18:01:12.742200+0000","last_clean":"2024-03-26T18:01:12.742200+0000","last_became_active":"2024-03-26T17:59:47.228367+0000","last_became_peered":"2024-03-26T17:59:47.228367+0000","last_unstale":"2024-03-26T18:01:12.742200+0000","last_undegraded":"2024-03-26T18:01:12.742200+0000","last_fullsized":"2024-03-26T18:01:12.742200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:59:04.158938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.748706+0000","last_change":"2024-03-26T17:59:47.230877+0000","last_active":"2024-03-26T18:01:12.748706+0000","last_peered":"2024-03-26T18:01:12.748706+0000","last_clean":"2024-03-26T18:01:12.748706+0000","last_became_active":"2024-03-26T17:59:47.230478+0000","last_became_peered":"2024-03-26T17:59:47.230478+0000","last_unstale":"2024-03-26T18:01:12.748706+0000","last_undegraded":"2024-03-26T18:01:12.748706+0000","last_fullsized":"2024-03-26T18:01:12.748706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:46:25.867718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.751204+0000","last_change":"2024-03-26T18:00:44.593728+0000","last_active":"2024-03-26T18:01:12.751204+0000","last_peered":"2024-03-26T18:01:12.751204+0000","last_clean":"2024-03-26T18:01:12.751204+0000","last_became_active":"2024-03-26T18:00:44.593306+0000","last_became_peered":"2024-03-26T18:00:44.593306+0000","last_unstale":"2024-03-26T18:01:12.751204+0000","last_undegraded":"2024-03-26T18:01:12.751204+0000","last_fullsized":"2024-03-26T18:01:12.751204+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:06:40.142046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.750062+0000","last_change":"2024-03-26T17:59:47.227641+0000","last_active":"2024-03-26T18:01:12.750062+0000","last_peered":"2024-03-26T18:01:12.750062+0000","last_clean":"2024-03-26T18:01:12.750062+0000","last_became_active":"2024-03-26T17:59:47.226921+0000","last_became_peered":"2024-03-26T17:59:47.226921+0000","last_unstale":"2024-03-26T18:01:12.750062+0000","last_undegraded":"2024-03-26T18:01:12.750062+0000","last_fullsized":"2024-03-26T18:01:12.750062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:48.442562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743477+0000","last_change":"2024-03-26T17:59:47.232387+0000","last_active":"2024-03-26T18:01:12.743477+0000","last_peered":"2024-03-26T18:01:12.743477+0000","last_clean":"2024-03-26T18:01:12.743477+0000","last_became_active":"2024-03-26T17:59:47.231934+0000","last_became_peered":"2024-03-26T17:59:47.231934+0000","last_unstale":"2024-03-26T18:01:12.743477+0000","last_undegraded":"2024-03-26T18:01:12.743477+0000","last_fullsized":"2024-03-26T18:01:12.743477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:32.006136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.739750+0000","last_change":"2024-03-26T17:59:47.226178+0000","last_active":"2024-03-26T18:01:12.739750+0000","last_peered":"2024-03-26T18:01:12.739750+0000","last_clean":"2024-03-26T18:01:12.739750+0000","last_became_active":"2024-03-26T17:59:47.224692+0000","last_became_peered":"2024-03-26T17:59:47.224692+0000","last_unstale":"2024-03-26T18:01:12.739750+0000","last_undegraded":"2024-03-26T18:01:12.739750+0000","last_fullsized":"2024-03-26T18:01:12.739750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:18.617841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.743707+0000","last_change":"2024-03-26T17:59:47.238289+0000","last_active":"2024-03-26T18:01:12.743707+0000","last_peered":"2024-03-26T18:01:12.743707+0000","last_clean":"2024-03-26T18:01:12.743707+0000","last_became_active":"2024-03-26T17:59:47.237904+0000","last_became_peered":"2024-03-26T17:59:47.237904+0000","last_unstale":"2024-03-26T18:01:12.743707+0000","last_undegraded":"2024-03-26T18:01:12.743707+0000","last_fullsized":"2024-03-26T18:01:12.743707+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:57:42.695335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.738434+0000","last_change":"2024-03-26T17:59:47.234780+0000","last_active":"2024-03-26T18:01:12.738434+0000","last_peered":"2024-03-26T18:01:12.738434+0000","last_clean":"2024-03-26T18:01:12.738434+0000","last_became_active":"2024-03-26T17:59:47.234478+0000","last_became_peered":"2024-03-26T17:59:47.234478+0000","last_unstale":"2024-03-26T18:01:12.738434+0000","last_undegraded":"2024-03-26T18:01:12.738434+0000","last_fullsized":"2024-03-26T18:01:12.738434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:57:43.675947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.746318+0000","last_change":"2024-03-26T17:59:47.240057+0000","last_active":"2024-03-26T18:01:12.746318+0000","last_peered":"2024-03-26T18:01:12.746318+0000","last_clean":"2024-03-26T18:01:12.746318+0000","last_became_active":"2024-03-26T17:59:47.239725+0000","last_became_peered":"2024-03-26T17:59:47.239725+0000","last_unstale":"2024-03-26T18:01:12.746318+0000","last_undegraded":"2024-03-26T18:01:12.746318+0000","last_fullsized":"2024-03-26T18:01:12.746318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:45.364403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.761051+0000","last_change":"2024-03-26T17:59:47.228059+0000","last_active":"2024-03-26T18:01:12.761051+0000","last_peered":"2024-03-26T18:01:12.761051+0000","last_clean":"2024-03-26T18:01:12.761051+0000","last_became_active":"2024-03-26T17:59:47.227666+0000","last_became_peered":"2024-03-26T17:59:47.227666+0000","last_unstale":"2024-03-26T18:01:12.761051+0000","last_undegraded":"2024-03-26T18:01:12.761051+0000","last_fullsized":"2024-03-26T18:01:12.761051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:19:26.440271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.748425+0000","last_change":"2024-03-26T17:59:47.228705+0000","last_active":"2024-03-26T18:01:12.748425+0000","last_peered":"2024-03-26T18:01:12.748425+0000","last_clean":"2024-03-26T18:01:12.748425+0000","last_became_active":"2024-03-26T17:59:47.228425+0000","last_became_peered":"2024-03-26T17:59:47.228425+0000","last_unstale":"2024-03-26T18:01:12.748425+0000","last_undegraded":"2024-03-26T18:01:12.748425+0000","last_fullsized":"2024-03-26T18:01:12.748425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:43:32.055816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T18:01:12.742776+0000","last_change":"2024-03-26T17:59:47.232080+0000","last_active":"2024-03-26T18:01:12.742776+0000","last_peered":"2024-03-26T18:01:12.742776+0000","last_clean":"2024-03-26T18:01:12.742776+0000","last_became_active":"2024-03-26T17:59:47.231672+0000","last_became_peered":"2024-03-26T17:59:47.231672+0000","last_unstale":"2024-03-26T18:01:12.742776+0000","last_undegraded":"2024-03-26T18:01:12.742776+0000","last_fullsized":"2024-03-26T18:01:12.742776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:59:46.410828+0000","last_clean_scrub_stamp":"2024-03-26T17:59:46.410828+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:39.962065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250240,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70068,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":756,"num_read_kb":457,"num_write":252,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210204,"data_compressed":4800,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":504413,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612645,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32308,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712844,"statfs":{"total":95995035648,"available":95961952256,"internally_reserved":0,"allocated":1159168,"data_stored":1154343,"data_compressed":58496,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776170,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":508439,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":504721,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1101891,"data_compressed":56896,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":499847,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":563459,"data_compressed":22280,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":501303,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":564211,"data_compressed":22280,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":499555,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888131,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":504453,"data_compressed":20680,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1096941,"data_compressed":56896,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34580,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34580,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36216,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64334,"data_compressed":1600,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65086,"data_compressed":1600,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64850,"data_compressed":1600,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T18:01:23.409 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-26T18:01:23.409 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-26T18:01:23.409 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-26T18:01:23.410 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-26T18:01:23.668 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:24.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:24 smithi171 ceph-mon[36033]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:01:24.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:24 smithi171 ceph-mon[36033]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:01:24.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:24 smithi115 ceph-mon[36174]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:01:24.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:24 smithi115 ceph-mon[36174]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:01:24.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:24 smithi049 ceph-mon[32331]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T18:01:24.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:24 smithi049 ceph-mon[32331]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T18:01:25.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.432+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/3242075526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e481012b0 msgr2=0x7f6e48101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:25.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.432+0000 7f6e4fc6a700 1 --2- 172.21.15.49:0/3242075526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e481012b0 0x7f6e48101710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009990 tx=0x7f6e3c009ca0 comp rx=0 tx=0).stop 2024-03-26T18:01:25.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.432+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/3242075526 shutdown_connections 2024-03-26T18:01:25.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.432+0000 7f6e4fc6a700 1 --2- 172.21.15.49:0/3242075526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e48101c50 0x7f6e4810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:25.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.432+0000 7f6e4fc6a700 1 --2- 172.21.15.49:0/3242075526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e481072f0 0x7f6e481076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:25.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.432+0000 7f6e4fc6a700 1 --2- 172.21.15.49:0/3242075526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e481012b0 0x7f6e48101710 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:25.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.432+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/3242075526 >> 172.21.15.49:0/3242075526 conn(0x7f6e480781e0 msgr2=0x7f6e480785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:25.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.432+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/3242075526 shutdown_connections 2024-03-26T18:01:25.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.432+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/3242075526 wait complete. 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.433+0000 7f6e4fc6a700 1 Processor -- start 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.433+0000 7f6e4fc6a700 1 -- start start 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.433+0000 7f6e4fc6a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e481012b0 0x7f6e48199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.433+0000 7f6e4fc6a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e48101c50 0x7f6e48199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.433+0000 7f6e4fc6a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e481072f0 0x7f6e481a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.433+0000 7f6e4fc6a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6e48111570 con 0x7f6e48101c50 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.433+0000 7f6e4fc6a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6e481113f0 con 0x7f6e481012b0 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.433+0000 7f6e4fc6a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6e481116f0 con 0x7f6e481072f0 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4da06700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e481012b0 0x7f6e48199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4d205700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e48101c50 0x7f6e48199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:25.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4da06700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e481012b0 0x7f6e48199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:33122/0 (socket says 172.21.15.49:33122) 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4d205700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e48101c50 0x7f6e48199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58524/0 (socket says 172.21.15.49:58524) 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4da06700 1 -- 172.21.15.49:0/2309443777 learned_addr learned my addr 172.21.15.49:0/2309443777 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4e207700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e481072f0 0x7f6e481a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4da06700 1 -- 172.21.15.49:0/2309443777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e481072f0 msgr2=0x7f6e481a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4da06700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e481072f0 0x7f6e481a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4da06700 1 -- 172.21.15.49:0/2309443777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e48101c50 msgr2=0x7f6e48199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4da06700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e48101c50 0x7f6e48199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4da06700 1 -- 172.21.15.49:0/2309443777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e481a5010 con 0x7f6e481012b0 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4da06700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e481012b0 0x7f6e48199a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c370 tx=0x7f6e3800c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e46ffd700 1 -- 172.21.15.49:0/2309443777 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e380092e0 con 0x7f6e481012b0 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e46ffd700 1 -- 172.21.15.49:0/2309443777 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e3800cbe0 con 0x7f6e481012b0 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/2309443777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e481a51e0 con 0x7f6e481012b0 2024-03-26T18:01:25.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e46ffd700 1 -- 172.21.15.49:0/2309443777 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e380047e0 con 0x7f6e481012b0 2024-03-26T18:01:25.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.434+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/2309443777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e481a56f0 con 0x7f6e481012b0 2024-03-26T18:01:25.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.435+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/2309443777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e48069070 con 0x7f6e481012b0 2024-03-26T18:01:25.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.436+0000 7f6e46ffd700 1 -- 172.21.15.49:0/2309443777 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f6e3801a020 con 0x7f6e481012b0 2024-03-26T18:01:25.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.437+0000 7f6e46ffd700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6e300a2fb0 0x7f6e300a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:25.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.437+0000 7f6e4d205700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6e300a2fb0 0x7f6e300a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:25.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.437+0000 7f6e46ffd700 1 -- 172.21.15.49:0/2309443777 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f6e380cefd0 con 0x7f6e481012b0 2024-03-26T18:01:25.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.437+0000 7f6e4d205700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6e300a2fb0 0x7f6e300a5470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00b2e0 tx=0x7f6e3c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:25.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.442+0000 7f6e46ffd700 1 -- 172.21.15.49:0/2309443777 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e38010040 con 0x7f6e481012b0 2024-03-26T18:01:25.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.693+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/2309443777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6e4802dc00 con 0x7f6e481012b0 2024-03-26T18:01:25.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.694+0000 7f6e46ffd700 1 -- 172.21.15.49:0/2309443777 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6e3809bac0 con 0x7f6e481012b0 2024-03-26T18:01:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:25.676 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-26T18:01:25.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/2309443777 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6e300a2fb0 msgr2=0x7f6e300a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:25.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6e300a2fb0 0x7f6e300a5470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00b2e0 tx=0x7f6e3c005960 comp rx=0 tx=0).stop 2024-03-26T18:01:25.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/2309443777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e481012b0 msgr2=0x7f6e48199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e481012b0 0x7f6e48199a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c370 tx=0x7f6e3800c680 comp rx=0 tx=0).stop 2024-03-26T18:01:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/2309443777 shutdown_connections 2024-03-26T18:01:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6e300a2fb0 0x7f6e300a5470 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e481072f0 0x7f6e481a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e481012b0 0x7f6e48199a50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 --2- 172.21.15.49:0/2309443777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6e48101c50 0x7f6e48199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/2309443777 >> 172.21.15.49:0/2309443777 conn(0x7f6e480781e0 msgr2=0x7f6e48100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/2309443777 shutdown_connections 2024-03-26T18:01:25.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:25.696+0000 7f6e4fc6a700 1 -- 172.21.15.49:0/2309443777 wait complete. 2024-03-26T18:01:26.375 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-26T18:01:26.376 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:01:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.505+0000 7f1dc0f22640 1 -- 172.21.15.49:0/244836337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc1076a0 msgr2=0x7f1dbc109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.505+0000 7f1dc0f22640 1 --2- 172.21.15.49:0/244836337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc1076a0 0x7f1dbc109a90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ca0 tx=0x7f1db402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.505+0000 7f1dc0f22640 1 -- 172.21.15.49:0/244836337 shutdown_connections 2024-03-26T18:01:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.505+0000 7f1dc0f22640 1 --2- 172.21.15.49:0/244836337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dbc104d70 0x7f1dbc107160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.505+0000 7f1dc0f22640 1 --2- 172.21.15.49:0/244836337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc1076a0 0x7f1dbc109a90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.505+0000 7f1dc0f22640 1 --2- 172.21.15.49:0/244836337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc102440 0x7f1dbc104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.505+0000 7f1dc0f22640 1 -- 172.21.15.49:0/244836337 >> 172.21.15.49:0/244836337 conn(0x7f1dbc0fbdf0 msgr2=0x7f1dbc0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.505+0000 7f1dc0f22640 1 -- 172.21.15.49:0/244836337 shutdown_connections 2024-03-26T18:01:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.505+0000 7f1dc0f22640 1 -- 172.21.15.49:0/244836337 wait complete. 2024-03-26T18:01:26.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dc0f22640 1 Processor -- start 2024-03-26T18:01:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dc0f22640 1 -- start start 2024-03-26T18:01:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dc0f22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc102440 0x7f1dbc197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dc0f22640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc104d70 0x7f1dbc1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dc0f22640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dbc1076a0 0x7f1dbc19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dc0f22640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc10d590 con 0x7f1dbc102440 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dc0f22640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc10d410 con 0x7f1dbc104d70 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dc0f22640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc10d710 con 0x7f1dbc1076a0 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc102440 0x7f1dbc197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc102440 0x7f1dbc197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58556/0 (socket says 172.21.15.49:58556) 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dba59c640 1 -- 172.21.15.49:0/1557918277 learned_addr learned my addr 172.21.15.49:0/1557918277 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dbad9d640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dbc1076a0 0x7f1dbc19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1db9d9b640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc104d70 0x7f1dbc1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dba59c640 1 -- 172.21.15.49:0/1557918277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dbc1076a0 msgr2=0x7f1dbc19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dba59c640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dbc1076a0 0x7f1dbc19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dba59c640 1 -- 172.21.15.49:0/1557918277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc104d70 msgr2=0x7f1dbc1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dba59c640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc104d70 0x7f1dbc1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dba59c640 1 -- 172.21.15.49:0/1557918277 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc19f7c0 con 0x7f1dbc102440 2024-03-26T18:01:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.506+0000 7f1dba59c640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc102440 0x7f1dbc197ba0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e9f0 tx=0x7f1dac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.507+0000 7f1dab7fe640 1 -- 172.21.15.49:0/1557918277 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00cd80 con 0x7f1dbc102440 2024-03-26T18:01:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.507+0000 7f1dab7fe640 1 -- 172.21.15.49:0/1557918277 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dac010910 con 0x7f1dbc102440 2024-03-26T18:01:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.507+0000 7f1dab7fe640 1 -- 172.21.15.49:0/1557918277 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac018990 con 0x7f1dbc102440 2024-03-26T18:01:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.507+0000 7f1dc0f22640 1 -- 172.21.15.49:0/1557918277 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc19fab0 con 0x7f1dbc102440 2024-03-26T18:01:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.507+0000 7f1dc0f22640 1 -- 172.21.15.49:0/1557918277 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc19ffb0 con 0x7f1dbc102440 2024-03-26T18:01:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.508+0000 7f1dc0f22640 1 -- 172.21.15.49:0/1557918277 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dbc068b00 con 0x7f1dbc102440 2024-03-26T18:01:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.510+0000 7f1dab7fe640 1 -- 172.21.15.49:0/1557918277 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f1dac010ac0 con 0x7f1dbc102440 2024-03-26T18:01:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.510+0000 7f1dab7fe640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1da40a2f20 0x7f1da40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.510+0000 7f1db9d9b640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1da40a2f20 0x7f1da40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.510+0000 7f1dab7fe640 1 -- 172.21.15.49:0/1557918277 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f1dac0cfe60 con 0x7f1dbc102440 2024-03-26T18:01:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.511+0000 7f1db9d9b640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1da40a2f20 0x7f1da40a53e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc1990b0 tx=0x7f1db0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:26.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.514+0000 7f1dab7fe640 1 -- 172.21.15.49:0/1557918277 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1dac09c300 con 0x7f1dbc102440 2024-03-26T18:01:26.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:26 smithi171 ceph-mon[36033]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-26T18:01:26.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:26 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2309443777' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T18:01:26.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.699+0000 7f1dc0f22640 1 -- 172.21.15.49:0/1557918277 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dbc02d240 con 0x7f1dbc102440 2024-03-26T18:01:26.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.700+0000 7f1dab7fe640 1 -- 172.21.15.49:0/1557918277 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24578 (secure 0 0 0) 0x7f1dac0a03a0 con 0x7f1dbc102440 2024-03-26T18:01:26.683 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:26.683 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":92,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T18:00:44.575842+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 -- 172.21.15.49:0/1557918277 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1da40a2f20 msgr2=0x7f1da40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1da40a2f20 0x7f1da40a53e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc1990b0 tx=0x7f1db0005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 -- 172.21.15.49:0/1557918277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc102440 msgr2=0x7f1dbc197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc102440 0x7f1dbc197ba0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e9f0 tx=0x7f1dac00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 -- 172.21.15.49:0/1557918277 shutdown_connections 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f1da40a2f20 0x7f1da40a53e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dbc1076a0 0x7f1dbc19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dbc104d70 0x7f1dbc1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 --2- 172.21.15.49:0/1557918277 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dbc102440 0x7f1dbc197ba0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 -- 172.21.15.49:0/1557918277 >> 172.21.15.49:0/1557918277 conn(0x7f1dbc0fbdf0 msgr2=0x7f1dbc102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 -- 172.21.15.49:0/1557918277 shutdown_connections 2024-03-26T18:01:26.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.702+0000 7f1dc0f22640 1 -- 172.21.15.49:0/1557918277 wait complete. 2024-03-26T18:01:26.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:26 smithi049 ceph-mon[32331]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-26T18:01:26.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:26 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2309443777' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T18:01:26.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:59:45.382542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:59:46.221139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T17:59:49.890383+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T18:01:26.697 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:01:26.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:26 smithi115 ceph-mon[36174]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-26T18:01:26.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:26 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2309443777' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T18:01:26.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.826+0000 7fd868d37640 1 -- 172.21.15.49:0/2948359537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864103530 msgr2=0x7fd864109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.826+0000 7fd868d37640 1 --2- 172.21.15.49:0/2948359537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864103530 0x7fd864109dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009ca0 tx=0x7fd85c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.826+0000 7fd868d37640 1 -- 172.21.15.49:0/2948359537 shutdown_connections 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.826+0000 7fd868d37640 1 --2- 172.21.15.49:0/2948359537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864103530 0x7fd864109dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.826+0000 7fd868d37640 1 --2- 172.21.15.49:0/2948359537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd864102b70 0x7fd864102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.826+0000 7fd868d37640 1 --2- 172.21.15.49:0/2948359537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8640ffba0 0x7fd8640fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.826+0000 7fd868d37640 1 -- 172.21.15.49:0/2948359537 >> 172.21.15.49:0/2948359537 conn(0x7fd8640fb4d0 msgr2=0x7fd8640fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.826+0000 7fd868d37640 1 -- 172.21.15.49:0/2948359537 shutdown_connections 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.826+0000 7fd868d37640 1 -- 172.21.15.49:0/2948359537 wait complete. 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd868d37640 1 Processor -- start 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd868d37640 1 -- start start 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd868d37640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8640ffba0 0x7fd8641024e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd868d37640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864102b70 0x7fd864100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd868d37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd864103530 0x7fd864101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd868d37640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd86410d570 con 0x7fd864103530 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd868d37640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd86410d3f0 con 0x7fd864102b70 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd868d37640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd86410d6f0 con 0x7fd8640ffba0 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd86259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8640ffba0 0x7fd8641024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd862d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd864103530 0x7fd864101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd86259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8640ffba0 0x7fd8641024e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:57352/0 (socket says 172.21.15.49:57352) 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd86259c640 1 -- 172.21.15.49:0/1267070000 learned_addr learned my addr 172.21.15.49:0/1267070000 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd861d9b640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864102b70 0x7fd864100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd86259c640 1 -- 172.21.15.49:0/1267070000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864102b70 msgr2=0x7fd864100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd86259c640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864102b70 0x7fd864100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd86259c640 1 -- 172.21.15.49:0/1267070000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd864103530 msgr2=0x7fd864101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd86259c640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd864103530 0x7fd864101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd86259c640 1 -- 172.21.15.49:0/1267070000 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8641018f0 con 0x7fd8640ffba0 2024-03-26T18:01:26.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd862d9d640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd864103530 0x7fd864101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:26.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.827+0000 7fd861d9b640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864102b70 0x7fd864100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:26.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.828+0000 7fd86259c640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8640ffba0 0x7fd8641024e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00e9c0 tx=0x7fd84c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:26.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.828+0000 7fd85b7fe640 1 -- 172.21.15.49:0/1267070000 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00cd80 con 0x7fd8640ffba0 2024-03-26T18:01:26.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.828+0000 7fd868d37640 1 -- 172.21.15.49:0/1267070000 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8641a1c50 con 0x7fd8640ffba0 2024-03-26T18:01:26.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.828+0000 7fd85b7fe640 1 -- 172.21.15.49:0/1267070000 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84c0035f0 con 0x7fd8640ffba0 2024-03-26T18:01:26.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.828+0000 7fd85b7fe640 1 -- 172.21.15.49:0/1267070000 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c010630 con 0x7fd8640ffba0 2024-03-26T18:01:26.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.828+0000 7fd868d37640 1 -- 172.21.15.49:0/1267070000 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8641a21a0 con 0x7fd8640ffba0 2024-03-26T18:01:26.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.829+0000 7fd868d37640 1 -- 172.21.15.49:0/1267070000 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd864108c40 con 0x7fd8640ffba0 2024-03-26T18:01:26.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.830+0000 7fd85b7fe640 1 -- 172.21.15.49:0/1267070000 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fd84c0037a0 con 0x7fd8640ffba0 2024-03-26T18:01:26.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.831+0000 7fd85b7fe640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd8500a2f20 0x7fd8500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.831+0000 7fd861d9b640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd8500a2f20 0x7fd8500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.831+0000 7fd85b7fe640 1 -- 172.21.15.49:0/1267070000 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fd84c0cff90 con 0x7fd8640ffba0 2024-03-26T18:01:26.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.831+0000 7fd861d9b640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd8500a2f20 0x7fd8500a53e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd864102250 tx=0x7fd854005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:26.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:26.834+0000 7fd85b7fe640 1 -- 172.21.15.49:0/1267070000 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd84c09ca60 con 0x7fd8640ffba0 2024-03-26T18:01:26.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.017+0000 7fd868d37640 1 -- 172.21.15.49:0/1267070000 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd864062560 con 0x7fd8640ffba0 2024-03-26T18:01:27.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.018+0000 7fd85b7fe640 1 -- 172.21.15.49:0/1267070000 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fd84c09c360 con 0x7fd8640ffba0 2024-03-26T18:01:27.000 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T18:01:27.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 -- 172.21.15.49:0/1267070000 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd8500a2f20 msgr2=0x7fd8500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd8500a2f20 0x7fd8500a53e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd864102250 tx=0x7fd854005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 -- 172.21.15.49:0/1267070000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8640ffba0 msgr2=0x7fd8641024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8640ffba0 0x7fd8641024e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00e9c0 tx=0x7fd84c00ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 -- 172.21.15.49:0/1267070000 shutdown_connections 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fd8500a2f20 0x7fd8500a53e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd864103530 0x7fd864101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864102b70 0x7fd864100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 --2- 172.21.15.49:0/1267070000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8640ffba0 0x7fd8641024e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 -- 172.21.15.49:0/1267070000 >> 172.21.15.49:0/1267070000 conn(0x7fd8640fb4d0 msgr2=0x7fd864105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 -- 172.21.15.49:0/1267070000 shutdown_connections 2024-03-26T18:01:27.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.020+0000 7fd868d37640 1 -- 172.21.15.49:0/1267070000 wait complete. 2024-03-26T18:01:27.013 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:01:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.142+0000 7f6994036640 1 -- 172.21.15.49:0/3052823787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f698c100ab0 msgr2=0x7f698c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.142+0000 7f6994036640 1 --2- 172.21.15.49:0/3052823787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f698c100ab0 0x7f698c100f10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f697c009900 tx=0x7f697c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.142+0000 7f6994036640 1 -- 172.21.15.49:0/3052823787 shutdown_connections 2024-03-26T18:01:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.142+0000 7f6994036640 1 --2- 172.21.15.49:0/3052823787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f698c101450 0x7f698c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.142+0000 7f6994036640 1 --2- 172.21.15.49:0/3052823787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f698c100ab0 0x7f698c100f10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.142+0000 7f6994036640 1 --2- 172.21.15.49:0/3052823787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c100150 0x7f698c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.142+0000 7f6994036640 1 -- 172.21.15.49:0/3052823787 >> 172.21.15.49:0/3052823787 conn(0x7f698c0fbd70 msgr2=0x7f698c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.142+0000 7f6994036640 1 -- 172.21.15.49:0/3052823787 shutdown_connections 2024-03-26T18:01:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.142+0000 7f6994036640 1 -- 172.21.15.49:0/3052823787 wait complete. 2024-03-26T18:01:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6994036640 1 Processor -- start 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6994036640 1 -- start start 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6994036640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c100150 0x7f698c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6994036640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f698c100ab0 0x7f698c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6994036640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f698c101450 0x7f698c1a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6994036640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f698c10f750 con 0x7f698c100150 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6994036640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f698c10f5d0 con 0x7f698c101450 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6994036640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f698c10f8d0 con 0x7f698c100ab0 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6991dd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c100150 0x7f698c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6991dd2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c100150 0x7f698c199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58598/0 (socket says 172.21.15.49:58598) 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f6991dd2640 1 -- 172.21.15.49:0/4110241884 learned_addr learned my addr 172.21.15.49:0/4110241884 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f69925d3640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f698c101450 0x7f698c1a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.143+0000 7f69915d1640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f698c100ab0 0x7f698c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6991dd2640 1 -- 172.21.15.49:0/4110241884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f698c100ab0 msgr2=0x7f698c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6991dd2640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f698c100ab0 0x7f698c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6991dd2640 1 -- 172.21.15.49:0/4110241884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f698c101450 msgr2=0x7f698c1a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6991dd2640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f698c101450 0x7f698c1a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6991dd2640 1 -- 172.21.15.49:0/4110241884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c1a1a10 con 0x7f698c100150 2024-03-26T18:01:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6991dd2640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c100150 0x7f698c199df0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f698800e9f0 tx=0x7f698800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6982ffd640 1 -- 172.21.15.49:0/4110241884 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698800cd80 con 0x7f698c100150 2024-03-26T18:01:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6994036640 1 -- 172.21.15.49:0/4110241884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c1a1d00 con 0x7f698c100150 2024-03-26T18:01:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6982ffd640 1 -- 172.21.15.49:0/4110241884 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6988010910 con 0x7f698c100150 2024-03-26T18:01:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6982ffd640 1 -- 172.21.15.49:0/4110241884 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880188f0 con 0x7f698c100150 2024-03-26T18:01:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.144+0000 7f6994036640 1 -- 172.21.15.49:0/4110241884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c1a21c0 con 0x7f698c100150 2024-03-26T18:01:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.145+0000 7f6994036640 1 -- 172.21.15.49:0/4110241884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c10aea0 con 0x7f698c100150 2024-03-26T18:01:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.146+0000 7f6982ffd640 1 -- 172.21.15.49:0/4110241884 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f6988010ac0 con 0x7f698c100150 2024-03-26T18:01:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.147+0000 7f6982ffd640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f69780a2f20 0x7f69780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.147+0000 7f69915d1640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f69780a2f20 0x7f69780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.147+0000 7f6982ffd640 1 -- 172.21.15.49:0/4110241884 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f69880cfea0 con 0x7f698c100150 2024-03-26T18:01:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.147+0000 7f69915d1640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f69780a2f20 0x7f69780a53e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f697c02c7b0 tx=0x7f697c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:27.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.150+0000 7f6982ffd640 1 -- 172.21.15.49:0/4110241884 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f698809c340 con 0x7f698c100150 2024-03-26T18:01:27.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.334+0000 7f6994036640 1 -- 172.21.15.49:0/4110241884 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f698c02d390 con 0x7f698c100150 2024-03-26T18:01:27.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.335+0000 7f6982ffd640 1 -- 172.21.15.49:0/4110241884 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f69880a03e0 con 0x7f698c100150 2024-03-26T18:01:27.317 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:01:27.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 -- 172.21.15.49:0/4110241884 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f69780a2f20 msgr2=0x7f69780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f69780a2f20 0x7f69780a53e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f697c02c7b0 tx=0x7f697c005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 -- 172.21.15.49:0/4110241884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c100150 msgr2=0x7f698c199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c100150 0x7f698c199df0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f698800e9f0 tx=0x7f698800eeb0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 -- 172.21.15.49:0/4110241884 shutdown_connections 2024-03-26T18:01:27.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f69780a2f20 0x7f69780a53e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f698c100ab0 0x7f698c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f698c101450 0x7f698c1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 --2- 172.21.15.49:0/4110241884 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f698c100150 0x7f698c199df0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 -- 172.21.15.49:0/4110241884 >> 172.21.15.49:0/4110241884 conn(0x7f698c0fbd70 msgr2=0x7f698c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:27.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 -- 172.21.15.49:0/4110241884 shutdown_connections 2024-03-26T18:01:27.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.337+0000 7f6994036640 1 -- 172.21.15.49:0/4110241884 wait complete. 2024-03-26T18:01:27.330 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:01:27.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.459+0000 7f4705dc4640 1 -- 172.21.15.49:0/2312752886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47001019b0 msgr2=0x7f4700101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.459+0000 7f4705dc4640 1 --2- 172.21.15.49:0/2312752886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47001019b0 0x7f4700101e30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009900 tx=0x7f46f402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:27.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.459+0000 7f4705dc4640 1 -- 172.21.15.49:0/2312752886 shutdown_connections 2024-03-26T18:01:27.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.459+0000 7f4705dc4640 1 --2- 172.21.15.49:0/2312752886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4700102370 0x7f4700108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.459+0000 7f4705dc4640 1 --2- 172.21.15.49:0/2312752886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47001019b0 0x7f4700101e30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.459+0000 7f4705dc4640 1 --2- 172.21.15.49:0/2312752886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47001007c0 0x7f4700100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.459+0000 7f4705dc4640 1 -- 172.21.15.49:0/2312752886 >> 172.21.15.49:0/2312752886 conn(0x7f47000fbd50 msgr2=0x7f47000fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:27.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.459+0000 7f4705dc4640 1 -- 172.21.15.49:0/2312752886 shutdown_connections 2024-03-26T18:01:27.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.459+0000 7f4705dc4640 1 -- 172.21.15.49:0/2312752886 wait complete. 2024-03-26T18:01:27.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.459+0000 7f4705dc4640 1 Processor -- start 2024-03-26T18:01:27.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f4705dc4640 1 -- start start 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f4705dc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47001007c0 0x7f4700197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f4705dc4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47001019b0 0x7f4700198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f4705dc4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4700102370 0x7f470019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f4705dc4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f470010c440 con 0x7f47001007c0 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f4705dc4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f470010c2c0 con 0x7f4700102370 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f4705dc4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f470010c5c0 con 0x7f47001019b0 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46ff7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47001007c0 0x7f4700197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46ff7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47001007c0 0x7f4700197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58612/0 (socket says 172.21.15.49:58612) 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46ff7fe640 1 -- 172.21.15.49:0/680123083 learned_addr learned my addr 172.21.15.49:0/680123083 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46fffff640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4700102370 0x7f470019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46feffd640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47001019b0 0x7f4700198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46ff7fe640 1 -- 172.21.15.49:0/680123083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47001019b0 msgr2=0x7f4700198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46ff7fe640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47001019b0 0x7f4700198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46ff7fe640 1 -- 172.21.15.49:0/680123083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4700102370 msgr2=0x7f470019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46ff7fe640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4700102370 0x7f470019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46ff7fe640 1 -- 172.21.15.49:0/680123083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470019f800 con 0x7f47001007c0 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46fffff640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4700102370 0x7f470019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.460+0000 7f46feffd640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47001019b0 0x7f4700198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.461+0000 7f46ff7fe640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47001007c0 0x7f4700197be0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e9c0 tx=0x7f46f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.461+0000 7f46fcff9640 1 -- 172.21.15.49:0/680123083 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f000cd80 con 0x7f47001007c0 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.461+0000 7f4705dc4640 1 -- 172.21.15.49:0/680123083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470019faf0 con 0x7f47001007c0 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.461+0000 7f46fcff9640 1 -- 172.21.15.49:0/680123083 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46f00035f0 con 0x7f47001007c0 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.461+0000 7f46fcff9640 1 -- 172.21.15.49:0/680123083 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0010690 con 0x7f47001007c0 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.461+0000 7f4705dc4640 1 -- 172.21.15.49:0/680123083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47001a0010 con 0x7f47001007c0 2024-03-26T18:01:27.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.462+0000 7f4705dc4640 1 -- 172.21.15.49:0/680123083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47001a01e0 con 0x7f47001007c0 2024-03-26T18:01:27.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.463+0000 7f46fcff9640 1 -- 172.21.15.49:0/680123083 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f46f00037a0 con 0x7f47001007c0 2024-03-26T18:01:27.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.464+0000 7f46fcff9640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f46e80a2f20 0x7f46e80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.464+0000 7f46feffd640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f46e80a2f20 0x7f46e80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.464+0000 7f46fcff9640 1 -- 172.21.15.49:0/680123083 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f46f00cfb80 con 0x7f47001007c0 2024-03-26T18:01:27.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.464+0000 7f46feffd640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f46e80a2f20 0x7f46e80a53e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46f402c7b0 tx=0x7f46f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:27.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.468+0000 7f46fcff9640 1 -- 172.21.15.49:0/680123083 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f46f009c700 con 0x7f47001007c0 2024-03-26T18:01:27.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:27 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1557918277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:27.586 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:27 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1267070000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:27.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:27 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1557918277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:27.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:27 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1267070000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:27.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.656+0000 7f4705dc4640 1 -- 172.21.15.49:0/680123083 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f47001a03b0 con 0x7f47001007c0 2024-03-26T18:01:27.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.657+0000 7f46fcff9640 1 -- 172.21.15.49:0/680123083 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f46f009c000 con 0x7f47001007c0 2024-03-26T18:01:27.640 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-26T18:01:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 -- 172.21.15.49:0/680123083 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f46e80a2f20 msgr2=0x7f46e80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f46e80a2f20 0x7f46e80a53e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46f402c7b0 tx=0x7f46f4005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 -- 172.21.15.49:0/680123083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47001007c0 msgr2=0x7f4700197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47001007c0 0x7f4700197be0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e9c0 tx=0x7f46f000ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 -- 172.21.15.49:0/680123083 shutdown_connections 2024-03-26T18:01:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f46e80a2f20 0x7f46e80a53e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47001019b0 0x7f4700198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4700102370 0x7f470019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 --2- 172.21.15.49:0/680123083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47001007c0 0x7f4700197be0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 -- 172.21.15.49:0/680123083 >> 172.21.15.49:0/680123083 conn(0x7f47000fbd50 msgr2=0x7f47000fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 -- 172.21.15.49:0/680123083 shutdown_connections 2024-03-26T18:01:27.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.659+0000 7f4705dc4640 1 -- 172.21.15.49:0/680123083 wait complete. 2024-03-26T18:01:27.653 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T18:01:27.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:27 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1557918277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:27.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:27 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1267070000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:27.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.782+0000 7f28809d4640 1 -- 172.21.15.49:0/187265586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f287c102340 msgr2=0x7f287c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.782+0000 7f28809d4640 1 --2- 172.21.15.49:0/187265586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f287c102340 0x7f287c104730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f286c009ca0 tx=0x7f286c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.783+0000 7f28809d4640 1 -- 172.21.15.49:0/187265586 shutdown_connections 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.783+0000 7f28809d4640 1 --2- 172.21.15.49:0/187265586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f287c1075a0 0x7f287c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.783+0000 7f28809d4640 1 --2- 172.21.15.49:0/187265586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c104c70 0x7f287c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.783+0000 7f28809d4640 1 --2- 172.21.15.49:0/187265586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f287c102340 0x7f287c104730 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.783+0000 7f28809d4640 1 -- 172.21.15.49:0/187265586 >> 172.21.15.49:0/187265586 conn(0x7f287c0fbd50 msgr2=0x7f287c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.783+0000 7f28809d4640 1 -- 172.21.15.49:0/187265586 shutdown_connections 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.783+0000 7f28809d4640 1 -- 172.21.15.49:0/187265586 wait complete. 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.783+0000 7f28809d4640 1 Processor -- start 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.783+0000 7f28809d4640 1 -- start start 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f28809d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c102340 0x7f287c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f28809d4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f287c104c70 0x7f287c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f28809d4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f287c1075a0 0x7f287c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f28809d4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f287c10d560 con 0x7f287c102340 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f28809d4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f287c10d3e0 con 0x7f287c104c70 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f28809d4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f287c10d6e0 con 0x7f287c1075a0 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f287a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c102340 0x7f287c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f287a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c102340 0x7f287c197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58632/0 (socket says 172.21.15.49:58632) 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f287a59c640 1 -- 172.21.15.49:0/2006525096 learned_addr learned my addr 172.21.15.49:0/2006525096 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f2879d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f287c104c70 0x7f287c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f287ad9d640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f287c1075a0 0x7f287c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f287a59c640 1 -- 172.21.15.49:0/2006525096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f287c1075a0 msgr2=0x7f287c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f287a59c640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f287c1075a0 0x7f287c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f287a59c640 1 -- 172.21.15.49:0/2006525096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f287c104c70 msgr2=0x7f287c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f287a59c640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f287c104c70 0x7f287c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f287a59c640 1 -- 172.21.15.49:0/2006525096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c19f7f0 con 0x7f287c102340 2024-03-26T18:01:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f2879d9b640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f287c104c70 0x7f287c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f287a59c640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c102340 0x7f287c197bd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f286c037040 tx=0x7f286c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f28737fe640 1 -- 172.21.15.49:0/2006525096 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c041020 con 0x7f287c102340 2024-03-26T18:01:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f28809d4640 1 -- 172.21.15.49:0/2006525096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287c19fa80 con 0x7f287c102340 2024-03-26T18:01:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f28737fe640 1 -- 172.21.15.49:0/2006525096 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286c034cd0 con 0x7f287c102340 2024-03-26T18:01:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.784+0000 7f28737fe640 1 -- 172.21.15.49:0/2006525096 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c039040 con 0x7f287c102340 2024-03-26T18:01:27.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.785+0000 7f28809d4640 1 -- 172.21.15.49:0/2006525096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287c19ff70 con 0x7f287c102340 2024-03-26T18:01:27.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.786+0000 7f28809d4640 1 -- 172.21.15.49:0/2006525096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287c0fd840 con 0x7f287c102340 2024-03-26T18:01:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.787+0000 7f28737fe640 1 -- 172.21.15.49:0/2006525096 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f286c044020 con 0x7f287c102340 2024-03-26T18:01:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.788+0000 7f28737fe640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f28680a2f20 0x7f28680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.788+0000 7f2879d9b640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f28680a2f20 0x7f28680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.788+0000 7f28737fe640 1 -- 172.21.15.49:0/2006525096 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f286c0f1600 con 0x7f287c102340 2024-03-26T18:01:27.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.788+0000 7f2879d9b640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f28680a2f20 0x7f28680a53e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f28640095f0 tx=0x7f2864005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:27.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.792+0000 7f28737fe640 1 -- 172.21.15.49:0/2006525096 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f286c04b020 con 0x7f287c102340 2024-03-26T18:01:27.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.975+0000 7f28809d4640 1 -- 172.21.15.49:0/2006525096 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f287c0624c0 con 0x7f287c102340 2024-03-26T18:01:27.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.976+0000 7f28737fe640 1 -- 172.21.15.49:0/2006525096 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f286c0be180 con 0x7f287c102340 2024-03-26T18:01:27.959 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:01:27.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 -- 172.21.15.49:0/2006525096 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f28680a2f20 msgr2=0x7f28680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f28680a2f20 0x7f28680a53e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f28640095f0 tx=0x7f2864005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:27.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 -- 172.21.15.49:0/2006525096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c102340 msgr2=0x7f287c197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:27.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c102340 0x7f287c197bd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f286c037040 tx=0x7f286c032f40 comp rx=0 tx=0).stop 2024-03-26T18:01:27.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 -- 172.21.15.49:0/2006525096 shutdown_connections 2024-03-26T18:01:27.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f28680a2f20 0x7f28680a53e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f287c1075a0 0x7f287c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f287c104c70 0x7f287c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 --2- 172.21.15.49:0/2006525096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f287c102340 0x7f287c197bd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:27.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 -- 172.21.15.49:0/2006525096 >> 172.21.15.49:0/2006525096 conn(0x7f287c0fbd50 msgr2=0x7f287c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:27.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 -- 172.21.15.49:0/2006525096 shutdown_connections 2024-03-26T18:01:27.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:27.978+0000 7f28809d4640 1 -- 172.21.15.49:0/2006525096 wait complete. 2024-03-26T18:01:27.972 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:01:28.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 -- 172.21.15.49:0/2688361931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e701023b0 msgr2=0x7f8e70108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 --2- 172.21.15.49:0/2688361931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e701023b0 0x7f8e70108c40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009ca0 tx=0x7f8e6802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 -- 172.21.15.49:0/2688361931 shutdown_connections 2024-03-26T18:01:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 --2- 172.21.15.49:0/2688361931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e701023b0 0x7f8e70108c40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 --2- 172.21.15.49:0/2688361931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e701019f0 0x7f8e70101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 --2- 172.21.15.49:0/2688361931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e70100800 0x7f8e70100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 -- 172.21.15.49:0/2688361931 >> 172.21.15.49:0/2688361931 conn(0x7f8e700fbd70 msgr2=0x7f8e700fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 -- 172.21.15.49:0/2688361931 shutdown_connections 2024-03-26T18:01:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 -- 172.21.15.49:0/2688361931 wait complete. 2024-03-26T18:01:28.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 Processor -- start 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.098+0000 7f8e77a14640 1 -- start start 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e77a14640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e70100800 0x7f8e70197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e77a14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e701019f0 0x7f8e70198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e77a14640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e701023b0 0x7f8e7019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e77a14640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010c480 con 0x7f8e701019f0 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e77a14640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010c300 con 0x7f8e70100800 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e77a14640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7010c600 con 0x7f8e701023b0 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e74faf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e701019f0 0x7f8e70198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e74faf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e701019f0 0x7f8e70198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58654/0 (socket says 172.21.15.49:58654) 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e74faf640 1 -- 172.21.15.49:0/3829609133 learned_addr learned my addr 172.21.15.49:0/3829609133 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e757b0640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e70100800 0x7f8e70197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e75fb1640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e701023b0 0x7f8e7019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e74faf640 1 -- 172.21.15.49:0/3829609133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e701023b0 msgr2=0x7f8e7019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e74faf640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e701023b0 0x7f8e7019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e74faf640 1 -- 172.21.15.49:0/3829609133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e70100800 msgr2=0x7f8e70197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e74faf640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e70100800 0x7f8e70197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e74faf640 1 -- 172.21.15.49:0/3829609133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7019f7e0 con 0x7f8e701019f0 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e74faf640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e701019f0 0x7f8e70198100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00ea60 tx=0x7f8e6c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e667fc640 1 -- 172.21.15.49:0/3829609133 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c00cde0 con 0x7f8e701019f0 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e667fc640 1 -- 172.21.15.49:0/3829609133 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c0035f0 con 0x7f8e701019f0 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e77a14640 1 -- 172.21.15.49:0/3829609133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7019fad0 con 0x7f8e701019f0 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.099+0000 7f8e667fc640 1 -- 172.21.15.49:0/3829609133 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c010670 con 0x7f8e701019f0 2024-03-26T18:01:28.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.100+0000 7f8e77a14640 1 -- 172.21.15.49:0/3829609133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e701a0020 con 0x7f8e701019f0 2024-03-26T18:01:28.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.100+0000 7f8e77a14640 1 -- 172.21.15.49:0/3829609133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e701a0250 con 0x7f8e701019f0 2024-03-26T18:01:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.101+0000 7f8e667fc640 1 -- 172.21.15.49:0/3829609133 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f8e6c0037a0 con 0x7f8e701019f0 2024-03-26T18:01:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.102+0000 7f8e667fc640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.102+0000 7f8e757b0640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.102+0000 7f8e667fc640 1 -- 172.21.15.49:0/3829609133 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f8e6c0cf9e0 con 0x7f8e701019f0 2024-03-26T18:01:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.103+0000 7f8e757b0640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8e70101850 tx=0x7f8e60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.106+0000 7f8e667fc640 1 -- 172.21.15.49:0/3829609133 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8e6c09c4d0 con 0x7f8e701019f0 2024-03-26T18:01:28.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.332+0000 7f8e77a14640 1 -- 172.21.15.49:0/3829609133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e701a04c0 con 0x7f8e701019f0 2024-03-26T18:01:28.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.332+0000 7f8e667fc640 1 -- 172.21.15.49:0/3829609133 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11943 (secure 0 0 0) 0x7f8e6c09bdd0 con 0x7f8e701019f0 2024-03-26T18:01:28.315 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:28.315 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:59:46.796381+0000","modified":"2024-03-26T18:00:09.450708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24503,"mds_2":24478,"mds_3":24428,"mds_4":24388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24478,24503,24388,24428,24533]},"id":1}]} 2024-03-26T18:01:28.316 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-03-26T18:01:28.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.334+0000 7f8e77a14640 1 -- 172.21.15.49:0/3829609133 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8e5c0a2f20 msgr2=0x7f8e5c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.334+0000 7f8e77a14640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8e70101850 tx=0x7f8e60005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:28.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.334+0000 7f8e77a14640 1 -- 172.21.15.49:0/3829609133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e701019f0 msgr2=0x7f8e70198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.334+0000 7f8e77a14640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e701019f0 0x7f8e70198100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00ea60 tx=0x7f8e6c00ef20 comp rx=0 tx=0).stop 2024-03-26T18:01:28.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.335+0000 7f8e77a14640 1 -- 172.21.15.49:0/3829609133 shutdown_connections 2024-03-26T18:01:28.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.335+0000 7f8e77a14640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f8e5c0a2f20 0x7f8e5c0a53e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.335+0000 7f8e77a14640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e701023b0 0x7f8e7019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.335+0000 7f8e77a14640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e701019f0 0x7f8e70198100 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.335+0000 7f8e77a14640 1 --2- 172.21.15.49:0/3829609133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e70100800 0x7f8e70197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.335+0000 7f8e77a14640 1 -- 172.21.15.49:0/3829609133 >> 172.21.15.49:0/3829609133 conn(0x7f8e700fbd70 msgr2=0x7f8e700fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:28.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.335+0000 7f8e77a14640 1 -- 172.21.15.49:0/3829609133 shutdown_connections 2024-03-26T18:01:28.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.335+0000 7f8e77a14640 1 -- 172.21.15.49:0/3829609133 wait complete. 2024-03-26T18:01:28.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:01:28.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 -- 172.21.15.49:0/3150149623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ab8103600 msgr2=0x7f4ab8109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 --2- 172.21.15.49:0/3150149623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ab8103600 0x7f4ab8109e90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009c10 tx=0x7f4ab002bef0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 -- 172.21.15.49:0/3150149623 shutdown_connections 2024-03-26T18:01:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 --2- 172.21.15.49:0/3150149623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ab8102c40 0x7f4ab81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 --2- 172.21.15.49:0/3150149623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ab8103600 0x7f4ab8109e90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 --2- 172.21.15.49:0/3150149623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80ffbd0 0x7f4ab80ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 -- 172.21.15.49:0/3150149623 >> 172.21.15.49:0/3150149623 conn(0x7f4ab80fb4e0 msgr2=0x7f4ab80fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 -- 172.21.15.49:0/3150149623 shutdown_connections 2024-03-26T18:01:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 -- 172.21.15.49:0/3150149623 wait complete. 2024-03-26T18:01:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 Processor -- start 2024-03-26T18:01:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.455+0000 7f4abe4ac640 1 -- start start 2024-03-26T18:01:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4abe4ac640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80ffbd0 0x7f4ab8197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4abe4ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ab8102c40 0x7f4ab81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4abe4ac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ab8103600 0x7f4ab819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4abe4ac640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab810d580 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4abe4ac640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab810d400 con 0x7f4ab8102c40 2024-03-26T18:01:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4abe4ac640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab810d700 con 0x7f4ab8103600 2024-03-26T18:01:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80ffbd0 0x7f4ab8197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80ffbd0 0x7f4ab8197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58680/0 (socket says 172.21.15.49:58680) 2024-03-26T18:01:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab7fff640 1 -- 172.21.15.49:0/2825756738 learned_addr learned my addr 172.21.15.49:0/2825756738 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab77fe640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ab8102c40 0x7f4ab81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4abca49640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ab8103600 0x7f4ab819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab7fff640 1 -- 172.21.15.49:0/2825756738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ab8103600 msgr2=0x7f4ab819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab7fff640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ab8103600 0x7f4ab819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab7fff640 1 -- 172.21.15.49:0/2825756738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ab8102c40 msgr2=0x7f4ab81980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab7fff640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ab8102c40 0x7f4ab81980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab7fff640 1 -- 172.21.15.49:0/2825756738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab819f7b0 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab77fe640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ab8102c40 0x7f4ab81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab7fff640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80ffbd0 0x7f4ab8197b90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00e9f0 tx=0x7f4aac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab57fa640 1 -- 172.21.15.49:0/2825756738 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00ccc0 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab57fa640 1 -- 172.21.15.49:0/2825756738 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aac0035f0 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4abe4ac640 1 -- 172.21.15.49:0/2825756738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab819faa0 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.456+0000 7f4ab57fa640 1 -- 172.21.15.49:0/2825756738 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00ce60 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.457+0000 7f4abe4ac640 1 -- 172.21.15.49:0/2825756738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab819fff0 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.457+0000 7f4abe4ac640 1 -- 172.21.15.49:0/2825756738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab8108d70 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.458+0000 7f4ab57fa640 1 -- 172.21.15.49:0/2825756738 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f4aac0037a0 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.459+0000 7f4ab57fa640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4aa00a2f20 0x7f4aa00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.459+0000 7f4ab77fe640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4aa00a2f20 0x7f4aa00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.460+0000 7f4ab57fa640 1 -- 172.21.15.49:0/2825756738 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f4aac0cf950 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.460+0000 7f4ab77fe640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4aa00a2f20 0x7f4aa00a53e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ab81990a0 tx=0x7f4aa8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:28.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.464+0000 7f4ab57fa640 1 -- 172.21.15.49:0/2825756738 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4aac09c440 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:28 smithi171 ceph-mon[36033]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-26T18:01:28.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:28 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4110241884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:28.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:28 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/680123083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:28.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:28 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2006525096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:28.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.650+0000 7f4abe4ac640 1 -- 172.21.15.49:0/2825756738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ab802d240 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.651+0000 7f4ab57fa640 1 -- 172.21.15.49:0/2825756738 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24578 (secure 0 0 0) 0x7f4aac09bd40 con 0x7f4ab80ffbd0 2024-03-26T18:01:28.634 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:28.634 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":92,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T18:00:44.575842+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:01:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 -- 172.21.15.49:0/2825756738 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4aa00a2f20 msgr2=0x7f4aa00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4aa00a2f20 0x7f4aa00a53e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ab81990a0 tx=0x7f4aa8005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 -- 172.21.15.49:0/2825756738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80ffbd0 msgr2=0x7f4ab8197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80ffbd0 0x7f4ab8197b90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00e9f0 tx=0x7f4aac00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 -- 172.21.15.49:0/2825756738 shutdown_connections 2024-03-26T18:01:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f4aa00a2f20 0x7f4aa00a53e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ab8103600 0x7f4ab819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ab8102c40 0x7f4ab81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 --2- 172.21.15.49:0/2825756738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4ab80ffbd0 0x7f4ab8197b90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 -- 172.21.15.49:0/2825756738 >> 172.21.15.49:0/2825756738 conn(0x7f4ab80fb4e0 msgr2=0x7f4ab8105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:28.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 -- 172.21.15.49:0/2825756738 shutdown_connections 2024-03-26T18:01:28.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.653+0000 7f4abe4ac640 1 -- 172.21.15.49:0/2825756738 wait complete. 2024-03-26T18:01:28.646 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:28 smithi049 ceph-mon[32331]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-26T18:01:28.646 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:28 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4110241884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:28.646 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:28 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/680123083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:28.646 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:28 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2006525096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:28.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:59:45.382542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:59:46.221139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T17:59:49.890383+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T18:01:28.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:01:28.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:28 smithi115 ceph-mon[36174]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-26T18:01:28.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:28 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4110241884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:28.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:28 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/680123083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:28.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:28 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2006525096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:28.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 -- 172.21.15.49:0/2918351779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f501018d0 msgr2=0x7f3f50101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 --2- 172.21.15.49:0/2918351779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f501018d0 0x7f3f50101cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009900 tx=0x7f3f4402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 -- 172.21.15.49:0/2918351779 shutdown_connections 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 --2- 172.21.15.49:0/2918351779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f50103480 0x7f3f50109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 --2- 172.21.15.49:0/2918351779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f50102ac0 0x7f3f50102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 --2- 172.21.15.49:0/2918351779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f501018d0 0x7f3f50101cf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 -- 172.21.15.49:0/2918351779 >> 172.21.15.49:0/2918351779 conn(0x7f3f500fce60 msgr2=0x7f3f500ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 -- 172.21.15.49:0/2918351779 shutdown_connections 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 -- 172.21.15.49:0/2918351779 wait complete. 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 Processor -- start 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 -- start start 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.777+0000 7f3f56fc6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f501018d0 0x7f3f50075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f56fc6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f50102ac0 0x7f3f50073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f56fc6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f50103480 0x7f3f500741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f56fc6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5010d550 con 0x7f3f501018d0 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f56fc6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5010d3d0 con 0x7f3f50103480 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f56fc6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5010d6d0 con 0x7f3f50102ac0 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f54d62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f501018d0 0x7f3f50075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f4ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f50102ac0 0x7f3f50073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f54d62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f501018d0 0x7f3f50075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:59988/0 (socket says 172.21.15.49:59988) 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f54d62640 1 -- 172.21.15.49:0/3794704609 learned_addr learned my addr 172.21.15.49:0/3794704609 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f55563640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f50103480 0x7f3f500741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f4ffff640 1 -- 172.21.15.49:0/3794704609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f50103480 msgr2=0x7f3f500741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f4ffff640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f50103480 0x7f3f500741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f4ffff640 1 -- 172.21.15.49:0/3794704609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f501018d0 msgr2=0x7f3f50075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f4ffff640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f501018d0 0x7f3f50075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f4ffff640 1 -- 172.21.15.49:0/3794704609 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f50074a80 con 0x7f3f50102ac0 2024-03-26T18:01:28.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f55563640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f50103480 0x7f3f500741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:28.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f54d62640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f501018d0 0x7f3f50075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:28.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f4ffff640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f50102ac0 0x7f3f50073cb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00e9f0 tx=0x7f3f3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:28.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f4dffb640 1 -- 172.21.15.49:0/3794704609 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00cd80 con 0x7f3f50102ac0 2024-03-26T18:01:28.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f56fc6640 1 -- 172.21.15.49:0/3794704609 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f501a1d70 con 0x7f3f50102ac0 2024-03-26T18:01:28.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f4dffb640 1 -- 172.21.15.49:0/3794704609 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f3c0035f0 con 0x7f3f50102ac0 2024-03-26T18:01:28.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.778+0000 7f3f4dffb640 1 -- 172.21.15.49:0/3794704609 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c010690 con 0x7f3f50102ac0 2024-03-26T18:01:28.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.779+0000 7f3f56fc6640 1 -- 172.21.15.49:0/3794704609 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f501a2280 con 0x7f3f50102ac0 2024-03-26T18:01:28.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.780+0000 7f3f56fc6640 1 -- 172.21.15.49:0/3794704609 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f50108c60 con 0x7f3f50102ac0 2024-03-26T18:01:28.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.781+0000 7f3f4dffb640 1 -- 172.21.15.49:0/3794704609 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f3f3c003b30 con 0x7f3f50102ac0 2024-03-26T18:01:28.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.781+0000 7f3f4dffb640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3f400a2fa0 0x7f3f400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:28.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.781+0000 7f3f54d62640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3f400a2fa0 0x7f3f400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:28.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.781+0000 7f3f4dffb640 1 -- 172.21.15.49:0/3794704609 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f3f3c0cf980 con 0x7f3f50102ac0 2024-03-26T18:01:28.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.782+0000 7f3f54d62640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3f400a2fa0 0x7f3f400a5460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3f4402c8e0 tx=0x7f3f44005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:28.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.787+0000 7f3f4dffb640 1 -- 172.21.15.49:0/3794704609 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3f3c09c470 con 0x7f3f50102ac0 2024-03-26T18:01:28.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.967+0000 7f3f56fc6640 1 -- 172.21.15.49:0/3794704609 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f50062560 con 0x7f3f50102ac0 2024-03-26T18:01:28.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.967+0000 7f3f4dffb640 1 -- 172.21.15.49:0/3794704609 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f3f3c09bd70 con 0x7f3f50102ac0 2024-03-26T18:01:28.950 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T18:01:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 -- 172.21.15.49:0/3794704609 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3f400a2fa0 msgr2=0x7f3f400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3f400a2fa0 0x7f3f400a5460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3f4402c8e0 tx=0x7f3f44005b50 comp rx=0 tx=0).stop 2024-03-26T18:01:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 -- 172.21.15.49:0/3794704609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f50102ac0 msgr2=0x7f3f50073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f50102ac0 0x7f3f50073cb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00e9f0 tx=0x7f3f3c00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 -- 172.21.15.49:0/3794704609 shutdown_connections 2024-03-26T18:01:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f3f400a2fa0 0x7f3f400a5460 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f50102ac0 0x7f3f50073cb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f50103480 0x7f3f500741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 --2- 172.21.15.49:0/3794704609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f501018d0 0x7f3f50075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:28.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 -- 172.21.15.49:0/3794704609 >> 172.21.15.49:0/3794704609 conn(0x7f3f500fce60 msgr2=0x7f3f50105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:28.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 -- 172.21.15.49:0/3794704609 shutdown_connections 2024-03-26T18:01:28.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:28.969+0000 7f3f56fc6640 1 -- 172.21.15.49:0/3794704609 wait complete. 2024-03-26T18:01:28.963 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:01:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.091+0000 7f91e9f1b640 1 -- 172.21.15.49:0/1568270915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91e4102360 msgr2=0x7f91e4104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.091+0000 7f91e9f1b640 1 --2- 172.21.15.49:0/1568270915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91e4102360 0x7f91e4104750 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009900 tx=0x7f91d802bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.091+0000 7f91e9f1b640 1 -- 172.21.15.49:0/1568270915 shutdown_connections 2024-03-26T18:01:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.091+0000 7f91e9f1b640 1 --2- 172.21.15.49:0/1568270915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91e4104c90 0x7f91e4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.091+0000 7f91e9f1b640 1 --2- 172.21.15.49:0/1568270915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e41075c0 0x7f91e4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.091+0000 7f91e9f1b640 1 --2- 172.21.15.49:0/1568270915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91e4102360 0x7f91e4104750 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.091+0000 7f91e9f1b640 1 -- 172.21.15.49:0/1568270915 >> 172.21.15.49:0/1568270915 conn(0x7f91e40fbd70 msgr2=0x7f91e40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.091+0000 7f91e9f1b640 1 -- 172.21.15.49:0/1568270915 shutdown_connections 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.091+0000 7f91e9f1b640 1 -- 172.21.15.49:0/1568270915 wait complete. 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e9f1b640 1 Processor -- start 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e9f1b640 1 -- start start 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e9f1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91e4102360 0x7f91e4197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e9f1b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91e4104c90 0x7f91e4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e9f1b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e41075c0 0x7f91e419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e9f1b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f91e410d590 con 0x7f91e4102360 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e9f1b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f91e410d410 con 0x7f91e41075c0 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e9f1b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f91e410d710 con 0x7f91e4104c90 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91e4102360 0x7f91e4197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91e4102360 0x7f91e4197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60004/0 (socket says 172.21.15.49:60004) 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.092+0000 7f91e37fe640 1 -- 172.21.15.49:0/3711790874 learned_addr learned my addr 172.21.15.49:0/3711790874 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:29.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e2ffd640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91e4104c90 0x7f91e4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e3fff640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e41075c0 0x7f91e419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e37fe640 1 -- 172.21.15.49:0/3711790874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91e4104c90 msgr2=0x7f91e4198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e37fe640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91e4104c90 0x7f91e4198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e37fe640 1 -- 172.21.15.49:0/3711790874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e41075c0 msgr2=0x7f91e419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e37fe640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e41075c0 0x7f91e419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e37fe640 1 -- 172.21.15.49:0/3711790874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e419f860 con 0x7f91e4102360 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e37fe640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91e4102360 0x7f91e4197c50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009230 tx=0x7f91d802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e0ff9640 1 -- 172.21.15.49:0/3711790874 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8041070 con 0x7f91e4102360 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e0ff9640 1 -- 172.21.15.49:0/3711790874 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d8004450 con 0x7f91e4102360 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e9f1b640 1 -- 172.21.15.49:0/3711790874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e419faf0 con 0x7f91e4102360 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e0ff9640 1 -- 172.21.15.49:0/3711790874 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8003e20 con 0x7f91e4102360 2024-03-26T18:01:29.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.093+0000 7f91e9f1b640 1 -- 172.21.15.49:0/3711790874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e419ffe0 con 0x7f91e4102360 2024-03-26T18:01:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.094+0000 7f91e9f1b640 1 -- 172.21.15.49:0/3711790874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e40fd860 con 0x7f91e4102360 2024-03-26T18:01:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.095+0000 7f91e0ff9640 1 -- 172.21.15.49:0/3711790874 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f91d8044070 con 0x7f91e4102360 2024-03-26T18:01:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.096+0000 7f91e0ff9640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f91cc0a2f20 0x7f91cc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.096+0000 7f91e2ffd640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f91cc0a2f20 0x7f91cc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.096+0000 7f91e0ff9640 1 -- 172.21.15.49:0/3711790874 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f91d80f1320 con 0x7f91e4102360 2024-03-26T18:01:29.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.096+0000 7f91e2ffd640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f91cc0a2f20 0x7f91cc0a53e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f91e406a9f0 tx=0x7f91d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:29.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.100+0000 7f91e0ff9640 1 -- 172.21.15.49:0/3711790874 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f91d80bd7c0 con 0x7f91e4102360 2024-03-26T18:01:29.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.283+0000 7f91e9f1b640 1 -- 172.21.15.49:0/3711790874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f91e40624c0 con 0x7f91e4102360 2024-03-26T18:01:29.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.284+0000 7f91e0ff9640 1 -- 172.21.15.49:0/3711790874 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f91d80c1860 con 0x7f91e4102360 2024-03-26T18:01:29.267 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:01:29.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 -- 172.21.15.49:0/3711790874 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f91cc0a2f20 msgr2=0x7f91cc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f91cc0a2f20 0x7f91cc0a53e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f91e406a9f0 tx=0x7f91d4005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:29.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 -- 172.21.15.49:0/3711790874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91e4102360 msgr2=0x7f91e4197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91e4102360 0x7f91e4197c50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009230 tx=0x7f91d802c650 comp rx=0 tx=0).stop 2024-03-26T18:01:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 -- 172.21.15.49:0/3711790874 shutdown_connections 2024-03-26T18:01:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f91cc0a2f20 0x7f91cc0a53e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f91e4104c90 0x7f91e4198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e41075c0 0x7f91e419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 --2- 172.21.15.49:0/3711790874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f91e4102360 0x7f91e4197c50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 -- 172.21.15.49:0/3711790874 >> 172.21.15.49:0/3711790874 conn(0x7f91e40fbd70 msgr2=0x7f91e4102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 -- 172.21.15.49:0/3711790874 shutdown_connections 2024-03-26T18:01:29.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.287+0000 7f91e9f1b640 1 -- 172.21.15.49:0/3711790874 wait complete. 2024-03-26T18:01:29.280 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:01:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.409+0000 7f7c601cd640 1 -- 172.21.15.49:0/2308110179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c581007e0 msgr2=0x7f7c58100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.409+0000 7f7c601cd640 1 --2- 172.21.15.49:0/2308110179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c581007e0 0x7f7c58100c00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009900 tx=0x7f7c4802bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.409+0000 7f7c601cd640 1 -- 172.21.15.49:0/2308110179 shutdown_connections 2024-03-26T18:01:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.409+0000 7f7c601cd640 1 --2- 172.21.15.49:0/2308110179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c581019d0 0x7f7c58101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.409+0000 7f7c601cd640 1 --2- 172.21.15.49:0/2308110179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c58102390 0x7f7c58108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.409+0000 7f7c601cd640 1 --2- 172.21.15.49:0/2308110179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c581007e0 0x7f7c58100c00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.409+0000 7f7c601cd640 1 -- 172.21.15.49:0/2308110179 >> 172.21.15.49:0/2308110179 conn(0x7f7c580fbd70 msgr2=0x7f7c580fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.409+0000 7f7c601cd640 1 -- 172.21.15.49:0/2308110179 shutdown_connections 2024-03-26T18:01:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.409+0000 7f7c601cd640 1 -- 172.21.15.49:0/2308110179 wait complete. 2024-03-26T18:01:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c601cd640 1 Processor -- start 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c601cd640 1 -- start start 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c601cd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c581007e0 0x7f7c5806aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c601cd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c581019d0 0x7f7c5819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c601cd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c58102390 0x7f7c5819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c601cd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5810c450 con 0x7f7c581019d0 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c601cd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5810c2d0 con 0x7f7c581007e0 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c601cd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5810c5d0 con 0x7f7c58102390 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5d768640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c581019d0 0x7f7c5819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5df69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c581007e0 0x7f7c5806aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5df69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c581007e0 0x7f7c5806aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:38182/0 (socket says 172.21.15.49:38182) 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5df69640 1 -- 172.21.15.49:0/4099283431 learned_addr learned my addr 172.21.15.49:0/4099283431 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5e76a640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c58102390 0x7f7c5819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5d768640 1 -- 172.21.15.49:0/4099283431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c58102390 msgr2=0x7f7c5819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5d768640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c58102390 0x7f7c5819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5d768640 1 -- 172.21.15.49:0/4099283431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c581007e0 msgr2=0x7f7c5806aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5d768640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c581007e0 0x7f7c5806aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5d768640 1 -- 172.21.15.49:0/4099283431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5819f8f0 con 0x7f7c581019d0 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.410+0000 7f7c5df69640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c581007e0 0x7f7c5806aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.411+0000 7f7c5d768640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c581019d0 0x7f7c5819c9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400e9f0 tx=0x7f7c5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.411+0000 7f7c4effd640 1 -- 172.21.15.49:0/4099283431 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5400ccc0 con 0x7f7c581019d0 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.411+0000 7f7c601cd640 1 -- 172.21.15.49:0/4099283431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5819fb80 con 0x7f7c581019d0 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.411+0000 7f7c4effd640 1 -- 172.21.15.49:0/4099283431 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c54005020 con 0x7f7c581019d0 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.411+0000 7f7c4effd640 1 -- 172.21.15.49:0/4099283431 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54004a00 con 0x7f7c581019d0 2024-03-26T18:01:29.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.411+0000 7f7c601cd640 1 -- 172.21.15.49:0/4099283431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c581a0080 con 0x7f7c581019d0 2024-03-26T18:01:29.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.412+0000 7f7c601cd640 1 -- 172.21.15.49:0/4099283431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c58107bd0 con 0x7f7c581019d0 2024-03-26T18:01:29.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.413+0000 7f7c4effd640 1 -- 172.21.15.49:0/4099283431 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f7c540051d0 con 0x7f7c581019d0 2024-03-26T18:01:29.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.414+0000 7f7c4effd640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7c440a2f20 0x7f7c440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.414+0000 7f7c5df69640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7c440a2f20 0x7f7c440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.414+0000 7f7c4effd640 1 -- 172.21.15.49:0/4099283431 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f7c540cf670 con 0x7f7c581019d0 2024-03-26T18:01:29.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.414+0000 7f7c5df69640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7c440a2f20 0x7f7c440a53e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7c4802c8e0 tx=0x7f7c48005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:29.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.419+0000 7f7c4effd640 1 -- 172.21.15.49:0/4099283431 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7c5409c160 con 0x7f7c581019d0 2024-03-26T18:01:29.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.604+0000 7f7c601cd640 1 -- 172.21.15.49:0/4099283431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c580624c0 con 0x7f7c581019d0 2024-03-26T18:01:29.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.604+0000 7f7c4effd640 1 -- 172.21.15.49:0/4099283431 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f7c5409ba60 con 0x7f7c581019d0 2024-03-26T18:01:29.587 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-26T18:01:29.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 -- 172.21.15.49:0/4099283431 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7c440a2f20 msgr2=0x7f7c440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7c440a2f20 0x7f7c440a53e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7c4802c8e0 tx=0x7f7c48005b50 comp rx=0 tx=0).stop 2024-03-26T18:01:29.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 -- 172.21.15.49:0/4099283431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c581019d0 msgr2=0x7f7c5819c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c581019d0 0x7f7c5819c9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400e9f0 tx=0x7f7c5400edf0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 -- 172.21.15.49:0/4099283431 shutdown_connections 2024-03-26T18:01:29.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7c440a2f20 0x7f7c440a53e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c58102390 0x7f7c5819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c581019d0 0x7f7c5819c9c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 --2- 172.21.15.49:0/4099283431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c581007e0 0x7f7c5806aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 -- 172.21.15.49:0/4099283431 >> 172.21.15.49:0/4099283431 conn(0x7f7c580fbd70 msgr2=0x7f7c58104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:29.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 -- 172.21.15.49:0/4099283431 shutdown_connections 2024-03-26T18:01:29.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.606+0000 7f7c601cd640 1 -- 172.21.15.49:0/4099283431 wait complete. 2024-03-26T18:01:29.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:29 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3829609133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:29.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:29 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2825756738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:29.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:29 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3794704609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:29.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:29 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3711790874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:29.600 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T18:01:29.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:29 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3829609133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:29.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:29 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2825756738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:29.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:29 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3794704609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:29.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:29 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3711790874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:29.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.729+0000 7f6b5b44b640 1 -- 172.21.15.49:0/1076906998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b54102ad0 msgr2=0x7f6b54102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.729+0000 7f6b5b44b640 1 --2- 172.21.15.49:0/1076906998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b54102ad0 0x7f6b54102f50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009900 tx=0x7f6b4402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:29.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.729+0000 7f6b5b44b640 1 -- 172.21.15.49:0/1076906998 shutdown_connections 2024-03-26T18:01:29.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.729+0000 7f6b5b44b640 1 --2- 172.21.15.49:0/1076906998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b54103490 0x7f6b54109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.729+0000 7f6b5b44b640 1 --2- 172.21.15.49:0/1076906998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b54102ad0 0x7f6b54102f50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.729+0000 7f6b5b44b640 1 --2- 172.21.15.49:0/1076906998 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b541018e0 0x7f6b54101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.729+0000 7f6b5b44b640 1 -- 172.21.15.49:0/1076906998 >> 172.21.15.49:0/1076906998 conn(0x7f6b540fce50 msgr2=0x7f6b540ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:29.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.729+0000 7f6b5b44b640 1 -- 172.21.15.49:0/1076906998 shutdown_connections 2024-03-26T18:01:29.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.729+0000 7f6b5b44b640 1 -- 172.21.15.49:0/1076906998 wait complete. 2024-03-26T18:01:29.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b5b44b640 1 Processor -- start 2024-03-26T18:01:29.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b5b44b640 1 -- start start 2024-03-26T18:01:29.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b5b44b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b541018e0 0x7f6b541959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b5b44b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b54102ad0 0x7f6b54195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b5b44b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b54103490 0x7f6b5419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b5b44b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5410b380 con 0x7f6b54103490 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b5b44b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5410b200 con 0x7f6b54102ad0 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b5b44b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5410b500 con 0x7f6b541018e0 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b599e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b54103490 0x7f6b5419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b599e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b54103490 0x7f6b5419cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60018/0 (socket says 172.21.15.49:60018) 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b599e8640 1 -- 172.21.15.49:0/3573936937 learned_addr learned my addr 172.21.15.49:0/3573936937 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b589e6640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b54102ad0 0x7f6b54195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b591e7640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b541018e0 0x7f6b541959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b599e8640 1 -- 172.21.15.49:0/3573936937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b541018e0 msgr2=0x7f6b541959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b599e8640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b541018e0 0x7f6b541959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b599e8640 1 -- 172.21.15.49:0/3573936937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b54102ad0 msgr2=0x7f6b54195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b599e8640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b54102ad0 0x7f6b54195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.730+0000 7f6b599e8640 1 -- 172.21.15.49:0/3573936937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5419d610 con 0x7f6b54103490 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.731+0000 7f6b599e8640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b54103490 0x7f6b5419cfa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00edb0 tx=0x7f6b4c012490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.731+0000 7f6b4a7fc640 1 -- 172.21.15.49:0/3573936937 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c01c030 con 0x7f6b54103490 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.731+0000 7f6b5b44b640 1 -- 172.21.15.49:0/3573936937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5419d900 con 0x7f6b54103490 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.731+0000 7f6b4a7fc640 1 -- 172.21.15.49:0/3573936937 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b4c012db0 con 0x7f6b54103490 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.731+0000 7f6b4a7fc640 1 -- 172.21.15.49:0/3573936937 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c018400 con 0x7f6b54103490 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.731+0000 7f6b5b44b640 1 -- 172.21.15.49:0/3573936937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5419ddd0 con 0x7f6b54103490 2024-03-26T18:01:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.732+0000 7f6b5b44b640 1 -- 172.21.15.49:0/3573936937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b54108c00 con 0x7f6b54103490 2024-03-26T18:01:29.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.733+0000 7f6b4a7fc640 1 -- 172.21.15.49:0/3573936937 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f6b4c01f070 con 0x7f6b54103490 2024-03-26T18:01:29.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.734+0000 7f6b4a7fc640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b400a2f20 0x7f6b400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:29.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.734+0000 7f6b591e7640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b400a2f20 0x7f6b400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:29.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.734+0000 7f6b4a7fc640 1 -- 172.21.15.49:0/3573936937 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f6b4c0d36b0 con 0x7f6b54103490 2024-03-26T18:01:29.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.734+0000 7f6b591e7640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b400a2f20 0x7f6b400a53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6b54102930 tx=0x7f6b50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:29.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.739+0000 7f6b4a7fc640 1 -- 172.21.15.49:0/3573936937 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6b4c0a01a0 con 0x7f6b54103490 2024-03-26T18:01:29.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:29 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3829609133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:29.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:29 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2825756738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:29.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:29 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3794704609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:29.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:29 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3711790874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.922+0000 7f6b5b44b640 1 -- 172.21.15.49:0/3573936937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6b540624c0 con 0x7f6b54103490 2024-03-26T18:01:29.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.923+0000 7f6b4a7fc640 1 -- 172.21.15.49:0/3573936937 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f6b4c09faa0 con 0x7f6b54103490 2024-03-26T18:01:29.904 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:01:29.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 -- 172.21.15.49:0/3573936937 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b400a2f20 msgr2=0x7f6b400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b400a2f20 0x7f6b400a53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6b54102930 tx=0x7f6b50005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:29.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 -- 172.21.15.49:0/3573936937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b54103490 msgr2=0x7f6b5419cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:29.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b54103490 0x7f6b5419cfa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00edb0 tx=0x7f6b4c012490 comp rx=0 tx=0).stop 2024-03-26T18:01:29.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 -- 172.21.15.49:0/3573936937 shutdown_connections 2024-03-26T18:01:29.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6b400a2f20 0x7f6b400a53e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b54103490 0x7f6b5419cfa0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b54102ad0 0x7f6b54195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 --2- 172.21.15.49:0/3573936937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b541018e0 0x7f6b541959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:29.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 -- 172.21.15.49:0/3573936937 >> 172.21.15.49:0/3573936937 conn(0x7f6b540fce50 msgr2=0x7f6b54105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:29.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 -- 172.21.15.49:0/3573936937 shutdown_connections 2024-03-26T18:01:29.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:29.925+0000 7f6b5b44b640 1 -- 172.21.15.49:0/3573936937 wait complete. 2024-03-26T18:01:29.918 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:01:30.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.044+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/106086035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6d41023e0 msgr2=0x7fa6d41047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.044+0000 7fa6d9b7e640 1 --2- 172.21.15.49:0/106086035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6d41023e0 0x7fa6d41047d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009900 tx=0x7fa6c402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.044+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/106086035 shutdown_connections 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.044+0000 7fa6d9b7e640 1 --2- 172.21.15.49:0/106086035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6d4107640 0x7fa6d4109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.044+0000 7fa6d9b7e640 1 --2- 172.21.15.49:0/106086035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d4104d10 0x7fa6d4107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.044+0000 7fa6d9b7e640 1 --2- 172.21.15.49:0/106086035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6d41023e0 0x7fa6d41047d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.044+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/106086035 >> 172.21.15.49:0/106086035 conn(0x7fa6d40fbdf0 msgr2=0x7fa6d40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.044+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/106086035 shutdown_connections 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.044+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/106086035 wait complete. 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d9b7e640 1 Processor -- start 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d9b7e640 1 -- start start 2024-03-26T18:01:30.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d9b7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d41023e0 0x7fa6d4197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d9b7e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6d4104d10 0x7fa6d4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d9b7e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6d4107640 0x7fa6d419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d9b7e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410d590 con 0x7fa6d41023e0 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d9b7e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410d410 con 0x7fa6d4104d10 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d9b7e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410d710 con 0x7fa6d4107640 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d41023e0 0x7fa6d4197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d41023e0 0x7fa6d4197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60028/0 (socket says 172.21.15.49:60028) 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d37fe640 1 -- 172.21.15.49:0/692539298 learned_addr learned my addr 172.21.15.49:0/692539298 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d3fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6d4107640 0x7fa6d419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.045+0000 7fa6d2ffd640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6d4104d10 0x7fa6d4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d37fe640 1 -- 172.21.15.49:0/692539298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6d4107640 msgr2=0x7fa6d419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d37fe640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6d4107640 0x7fa6d419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d37fe640 1 -- 172.21.15.49:0/692539298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6d4104d10 msgr2=0x7fa6d4198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d37fe640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6d4104d10 0x7fa6d4198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d37fe640 1 -- 172.21.15.49:0/692539298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d419f860 con 0x7fa6d41023e0 2024-03-26T18:01:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d37fe640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d41023e0 0x7fa6d4197c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009230 tx=0x7fa6c402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d0ff9640 1 -- 172.21.15.49:0/692539298 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4041070 con 0x7fa6d41023e0 2024-03-26T18:01:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/692539298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d419faf0 con 0x7fa6d41023e0 2024-03-26T18:01:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d0ff9640 1 -- 172.21.15.49:0/692539298 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4004450 con 0x7fa6d41023e0 2024-03-26T18:01:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d0ff9640 1 -- 172.21.15.49:0/692539298 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4003e20 con 0x7fa6d41023e0 2024-03-26T18:01:30.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.046+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/692539298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d419ffe0 con 0x7fa6d41023e0 2024-03-26T18:01:30.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.047+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/692539298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d40fd8e0 con 0x7fa6d41023e0 2024-03-26T18:01:30.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.048+0000 7fa6d0ff9640 1 -- 172.21.15.49:0/692539298 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fa6c4044070 con 0x7fa6d41023e0 2024-03-26T18:01:30.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.049+0000 7fa6d0ff9640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa6bc0a2f20 0x7fa6bc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.049+0000 7fa6d2ffd640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa6bc0a2f20 0x7fa6bc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.049+0000 7fa6d0ff9640 1 -- 172.21.15.49:0/692539298 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fa6c40f1320 con 0x7fa6d41023e0 2024-03-26T18:01:30.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.049+0000 7fa6d2ffd640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa6bc0a2f20 0x7fa6bc0a53e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa6d406aa50 tx=0x7fa6c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:30.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.053+0000 7fa6d0ff9640 1 -- 172.21.15.49:0/692539298 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa6c40bd7c0 con 0x7fa6d41023e0 2024-03-26T18:01:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.279+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/692539298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6d402d240 con 0x7fa6d41023e0 2024-03-26T18:01:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.279+0000 7fa6d0ff9640 1 -- 172.21.15.49:0/692539298 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11943 (secure 0 0 0) 0x7fa6c40c1860 con 0x7fa6d41023e0 2024-03-26T18:01:30.261 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:30.261 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:59:46.796381+0000","modified":"2024-03-26T18:00:09.450708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24503,"mds_2":24478,"mds_3":24428,"mds_4":24388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24478,24503,24388,24428,24533]},"id":1}]} 2024-03-26T18:01:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-03-26T18:01:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.281+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/692539298 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa6bc0a2f20 msgr2=0x7fa6bc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.281+0000 7fa6d9b7e640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa6bc0a2f20 0x7fa6bc0a53e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa6d406aa50 tx=0x7fa6c8005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.281+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/692539298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d41023e0 msgr2=0x7fa6d4197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.281+0000 7fa6d9b7e640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d41023e0 0x7fa6d4197c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009230 tx=0x7fa6c402c650 comp rx=0 tx=0).stop 2024-03-26T18:01:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.281+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/692539298 shutdown_connections 2024-03-26T18:01:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.281+0000 7fa6d9b7e640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa6bc0a2f20 0x7fa6bc0a53e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.281+0000 7fa6d9b7e640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6d4107640 0x7fa6d419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.281+0000 7fa6d9b7e640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6d4104d10 0x7fa6d4198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.281+0000 7fa6d9b7e640 1 --2- 172.21.15.49:0/692539298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa6d41023e0 0x7fa6d4197c50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.281+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/692539298 >> 172.21.15.49:0/692539298 conn(0x7fa6d40fbdf0 msgr2=0x7fa6d4102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.282+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/692539298 shutdown_connections 2024-03-26T18:01:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.282+0000 7fa6d9b7e640 1 -- 172.21.15.49:0/692539298 wait complete. 2024-03-26T18:01:30.274 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:01:30.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.402+0000 7f103910c640 1 -- 172.21.15.49:0/2652998241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1034101a50 msgr2=0x7f1034101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.402+0000 7f103910c640 1 --2- 172.21.15.49:0/2652998241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1034101a50 0x7f1034101ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f101c009900 tx=0x7f101c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:30.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.402+0000 7f103910c640 1 -- 172.21.15.49:0/2652998241 shutdown_connections 2024-03-26T18:01:30.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.402+0000 7f103910c640 1 --2- 172.21.15.49:0/2652998241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1034102410 0x7f1034108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.402+0000 7f103910c640 1 --2- 172.21.15.49:0/2652998241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1034101a50 0x7f1034101ed0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.402+0000 7f103910c640 1 --2- 172.21.15.49:0/2652998241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1034100860 0x7f1034100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.402+0000 7f103910c640 1 -- 172.21.15.49:0/2652998241 >> 172.21.15.49:0/2652998241 conn(0x7f10340fbdf0 msgr2=0x7f10340fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.402+0000 7f103910c640 1 -- 172.21.15.49:0/2652998241 shutdown_connections 2024-03-26T18:01:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.402+0000 7f103910c640 1 -- 172.21.15.49:0/2652998241 wait complete. 2024-03-26T18:01:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103910c640 1 Processor -- start 2024-03-26T18:01:30.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103910c640 1 -- start start 2024-03-26T18:01:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103910c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1034100860 0x7f1034197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103910c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1034101a50 0x7f1034198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103910c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1034102410 0x7f103419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103910c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f103410d590 con 0x7f1034102410 2024-03-26T18:01:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103910c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f103410d410 con 0x7f1034101a50 2024-03-26T18:01:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103910c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f103410d710 con 0x7f1034100860 2024-03-26T18:01:30.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1034102410 0x7f103419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103359e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1034102410 0x7f103419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60050/0 (socket says 172.21.15.49:60050) 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f102a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1034101a50 0x7f1034198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f103359e640 1 -- 172.21.15.49:0/2876729543 learned_addr learned my addr 172.21.15.49:0/2876729543 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.403+0000 7f1032d9d640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1034100860 0x7f1034197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f103359e640 1 -- 172.21.15.49:0/2876729543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1034100860 msgr2=0x7f1034197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f103359e640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1034100860 0x7f1034197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f103359e640 1 -- 172.21.15.49:0/2876729543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1034101a50 msgr2=0x7f1034198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f103359e640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1034101a50 0x7f1034198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f103359e640 1 -- 172.21.15.49:0/2876729543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103419f800 con 0x7f1034102410 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f1032d9d640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1034100860 0x7f1034197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f102a59c640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1034101a50 0x7f1034198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f103359e640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1034102410 0x7f103419f190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f102c00eb10 tx=0x7f102c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f1030d99640 1 -- 172.21.15.49:0/2876729543 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c004b20 con 0x7f1034102410 2024-03-26T18:01:30.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f1030d99640 1 -- 172.21.15.49:0/2876729543 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102c004cc0 con 0x7f1034102410 2024-03-26T18:01:30.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f103910c640 1 -- 172.21.15.49:0/2876729543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103419faf0 con 0x7f1034102410 2024-03-26T18:01:30.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f1030d99640 1 -- 172.21.15.49:0/2876729543 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c010760 con 0x7f1034102410 2024-03-26T18:01:30.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.404+0000 7f103910c640 1 -- 172.21.15.49:0/2876729543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10341a0010 con 0x7f1034102410 2024-03-26T18:01:30.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.405+0000 7f103910c640 1 -- 172.21.15.49:0/2876729543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1034068b00 con 0x7f1034102410 2024-03-26T18:01:30.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.406+0000 7f1030d99640 1 -- 172.21.15.49:0/2876729543 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f102c010900 con 0x7f1034102410 2024-03-26T18:01:30.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.407+0000 7f1030d99640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10200a2fa0 0x7f10200a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.407+0000 7f1032d9d640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10200a2fa0 0x7f10200a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.407+0000 7f1030d99640 1 -- 172.21.15.49:0/2876729543 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f102c0cfe40 con 0x7f1034102410 2024-03-26T18:01:30.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.407+0000 7f1032d9d640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10200a2fa0 0x7f10200a5460 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f10341018b0 tx=0x7f1024005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:30.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.411+0000 7f1030d99640 1 -- 172.21.15.49:0/2876729543 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f102c053740 con 0x7f1034102410 2024-03-26T18:01:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.597+0000 7f103910c640 1 -- 172.21.15.49:0/2876729543 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1034107bd0 con 0x7f1034102410 2024-03-26T18:01:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.598+0000 7f1030d99640 1 -- 172.21.15.49:0/2876729543 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24578 (secure 0 0 0) 0x7f1034107bd0 con 0x7f1034102410 2024-03-26T18:01:30.580 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:30.580 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":92,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T18:00:44.575842+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:01:30.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.600+0000 7f103910c640 1 -- 172.21.15.49:0/2876729543 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10200a2fa0 msgr2=0x7f10200a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.600+0000 7f103910c640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10200a2fa0 0x7f10200a5460 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f10341018b0 tx=0x7f1024005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:30.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.600+0000 7f103910c640 1 -- 172.21.15.49:0/2876729543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1034102410 msgr2=0x7f103419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.600+0000 7f103910c640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1034102410 0x7f103419f190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f102c00eb10 tx=0x7f102c00c2d0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.600+0000 7f103910c640 1 -- 172.21.15.49:0/2876729543 shutdown_connections 2024-03-26T18:01:30.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.600+0000 7f103910c640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10200a2fa0 0x7f10200a5460 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.600+0000 7f103910c640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1034102410 0x7f103419f190 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.600+0000 7f103910c640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1034101a50 0x7f1034198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.600+0000 7f103910c640 1 --2- 172.21.15.49:0/2876729543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1034100860 0x7f1034197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.600+0000 7f103910c640 1 -- 172.21.15.49:0/2876729543 >> 172.21.15.49:0/2876729543 conn(0x7f10340fbdf0 msgr2=0x7f1034104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.601+0000 7f103910c640 1 -- 172.21.15.49:0/2876729543 shutdown_connections 2024-03-26T18:01:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.601+0000 7f103910c640 1 -- 172.21.15.49:0/2876729543 wait complete. 2024-03-26T18:01:30.592 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:30 smithi049 ceph-mon[32331]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-26T18:01:30.592 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:30 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4099283431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:30.592 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:30 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3573936937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:30.592 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:30 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/692539298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:30.594 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:01:30.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:30 smithi171 ceph-mon[36033]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-26T18:01:30.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:30 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4099283431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:30.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:30 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3573936937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:30.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:30 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/692539298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:30.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 -- 172.21.15.49:0/1550604483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d10101a50 msgr2=0x7f7d10101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 --2- 172.21.15.49:0/1550604483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d10101a50 0x7f7d10101ed0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009900 tx=0x7f7d0402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:30.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 -- 172.21.15.49:0/1550604483 shutdown_connections 2024-03-26T18:01:30.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 --2- 172.21.15.49:0/1550604483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d10102410 0x7f7d10108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 --2- 172.21.15.49:0/1550604483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d10101a50 0x7f7d10101ed0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 --2- 172.21.15.49:0/1550604483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d10100860 0x7f7d10100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 -- 172.21.15.49:0/1550604483 >> 172.21.15.49:0/1550604483 conn(0x7f7d100fbdf0 msgr2=0x7f7d100fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 -- 172.21.15.49:0/1550604483 shutdown_connections 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 -- 172.21.15.49:0/1550604483 wait complete. 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 Processor -- start 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.721+0000 7f7d17310640 1 -- start start 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d17310640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d10100860 0x7f7d101047f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d17310640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d10101a50 0x7f7d10102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d17310640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d10102410 0x7f7d10103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d17310640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010c470 con 0x7f7d10101a50 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d17310640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010c2f0 con 0x7f7d10102410 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d17310640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010c5f0 con 0x7f7d10100860 2024-03-26T18:01:30.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d148ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d10101a50 0x7f7d10102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d148ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d10101a50 0x7f7d10102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60062/0 (socket says 172.21.15.49:60062) 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d148ab640 1 -- 172.21.15.49:0/1519262994 learned_addr learned my addr 172.21.15.49:0/1519262994 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d158ad640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d10102410 0x7f7d10103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d150ac640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d10100860 0x7f7d101047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d148ab640 1 -- 172.21.15.49:0/1519262994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d10100860 msgr2=0x7f7d101047f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d148ab640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d10100860 0x7f7d101047f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d148ab640 1 -- 172.21.15.49:0/1519262994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d10102410 msgr2=0x7f7d10103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d148ab640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d10102410 0x7f7d10103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d148ab640 1 -- 172.21.15.49:0/1519262994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d10103c00 con 0x7f7d10101a50 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d158ad640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d10102410 0x7f7d10103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d148ab640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d10101a50 0x7f7d10102e30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009230 tx=0x7f7d0402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d027fc640 1 -- 172.21.15.49:0/1519262994 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04041070 con 0x7f7d10101a50 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d027fc640 1 -- 172.21.15.49:0/1519262994 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d04004450 con 0x7f7d10101a50 2024-03-26T18:01:30.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d17310640 1 -- 172.21.15.49:0/1519262994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d101a1ca0 con 0x7f7d10101a50 2024-03-26T18:01:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d027fc640 1 -- 172.21.15.49:0/1519262994 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04003e20 con 0x7f7d10101a50 2024-03-26T18:01:30.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.722+0000 7f7d17310640 1 -- 172.21.15.49:0/1519262994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d101a2190 con 0x7f7d10101a50 2024-03-26T18:01:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.723+0000 7f7d17310640 1 -- 172.21.15.49:0/1519262994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d101a23c0 con 0x7f7d10101a50 2024-03-26T18:01:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.725+0000 7f7d027fc640 1 -- 172.21.15.49:0/1519262994 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f7d04044070 con 0x7f7d10101a50 2024-03-26T18:01:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.725+0000 7f7d027fc640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7cfc0a2fa0 0x7f7cfc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.725+0000 7f7d150ac640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7cfc0a2fa0 0x7f7cfc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.725+0000 7f7d027fc640 1 -- 172.21.15.49:0/1519262994 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f7d040f1200 con 0x7f7d10101a50 2024-03-26T18:01:30.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.726+0000 7f7d150ac640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7cfc0a2fa0 0x7f7cfc0a5460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7d101018b0 tx=0x7f7d0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:30.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.730+0000 7f7d027fc640 1 -- 172.21.15.49:0/1519262994 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7d040bd6a0 con 0x7f7d10101a50 2024-03-26T18:01:30.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:30 smithi115 ceph-mon[36174]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-26T18:01:30.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:30 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4099283431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:30.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:30 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3573936937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:30.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:30 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/692539298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:30.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.951+0000 7f7d17310640 1 -- 172.21.15.49:0/1519262994 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d101a2590 con 0x7f7d10101a50 2024-03-26T18:01:30.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.952+0000 7f7d027fc640 1 -- 172.21.15.49:0/1519262994 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11943 (secure 0 0 0) 0x7f7d040c1740 con 0x7f7d10101a50 2024-03-26T18:01:30.934 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:30.934 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:59:46.796381+0000","modified":"2024-03-26T18:00:09.450708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24503,"mds_2":24478,"mds_3":24428,"mds_4":24388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24478,24503,24388,24428,24533]},"id":1}]} 2024-03-26T18:01:30.934 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-03-26T18:01:30.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 -- 172.21.15.49:0/1519262994 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7cfc0a2fa0 msgr2=0x7f7cfc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7cfc0a2fa0 0x7f7cfc0a5460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7d101018b0 tx=0x7f7d0c005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:30.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 -- 172.21.15.49:0/1519262994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d10101a50 msgr2=0x7f7d10102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:30.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d10101a50 0x7f7d10102e30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009230 tx=0x7f7d0402c650 comp rx=0 tx=0).stop 2024-03-26T18:01:30.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 -- 172.21.15.49:0/1519262994 shutdown_connections 2024-03-26T18:01:30.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7cfc0a2fa0 0x7f7cfc0a5460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d10102410 0x7f7d10103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d10101a50 0x7f7d10102e30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 --2- 172.21.15.49:0/1519262994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d10100860 0x7f7d101047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:30.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 -- 172.21.15.49:0/1519262994 >> 172.21.15.49:0/1519262994 conn(0x7f7d100fbdf0 msgr2=0x7f7d100fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:30.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 -- 172.21.15.49:0/1519262994 shutdown_connections 2024-03-26T18:01:30.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:30.954+0000 7f7d17310640 1 -- 172.21.15.49:0/1519262994 wait complete. 2024-03-26T18:01:30.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T17:59:46.796381+0000', 'modified': '2024-03-26T18:00:09.450708+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24533, 'mds_1': 24503, 'mds_2': 24478, 'mds_3': 24428, 'mds_4': 24388}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6839/906046850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 906046850}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 906046850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6841/935071179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6840', 'nonce': 935071179}, {'type': 'v1', 'addr': '172.21.15.49:6841', 'nonce': 935071179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24388': {'gid': 24388, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.171:6833/2067087026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6832', 'nonce': 2067087026}, {'type': 'v1', 'addr': '172.21.15.171:6833', 'nonce': 2067087026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.115:6833/3604276420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 3604276420}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 3604276420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6835/3854985421', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 3854985421}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 3854985421}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/2428288879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 2428288879}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 2428288879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/4085026954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 4085026954}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 4085026954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.171:6839/1082475840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 1082475840}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 1082475840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.115:6837/1993258438', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 1993258438}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 1993258438}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6839/1435040687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 1435040687}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 1435040687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24533, 'qdb_cluster': [24478, 24503, 24388, 24428, 24533]} 2024-03-26T18:01:30.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-26T18:01:30.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-03-26T18:01:31.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.075+0000 7f5d997df640 1 -- 172.21.15.49:0/80075582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d94107660 msgr2=0x7f5d94109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.075+0000 7f5d997df640 1 --2- 172.21.15.49:0/80075582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d94107660 0x7f5d94109a50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009ca0 tx=0x7f5d8c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.075+0000 7f5d997df640 1 -- 172.21.15.49:0/80075582 shutdown_connections 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.075+0000 7f5d997df640 1 --2- 172.21.15.49:0/80075582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d94104d30 0x7f5d94107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.075+0000 7f5d997df640 1 --2- 172.21.15.49:0/80075582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d94107660 0x7f5d94109a50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.075+0000 7f5d997df640 1 --2- 172.21.15.49:0/80075582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d94102400 0x7f5d941047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.075+0000 7f5d997df640 1 -- 172.21.15.49:0/80075582 >> 172.21.15.49:0/80075582 conn(0x7f5d940fbd70 msgr2=0x7f5d940fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.075+0000 7f5d997df640 1 -- 172.21.15.49:0/80075582 shutdown_connections 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.075+0000 7f5d997df640 1 -- 172.21.15.49:0/80075582 wait complete. 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.075+0000 7f5d997df640 1 Processor -- start 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d997df640 1 -- start start 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d997df640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d94102400 0x7f5d94197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d997df640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d94104d30 0x7f5d94198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d997df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d94107660 0x7f5d9419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d997df640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9410d560 con 0x7f5d94107660 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d997df640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9410d3e0 con 0x7f5d94104d30 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d997df640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9410d6e0 con 0x7f5d94102400 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d937fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d94107660 0x7f5d9419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d927fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d94104d30 0x7f5d94198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d937fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d94107660 0x7f5d9419f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60070/0 (socket says 172.21.15.49:60070) 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d937fe640 1 -- 172.21.15.49:0/770066790 learned_addr learned my addr 172.21.15.49:0/770066790 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d94102400 0x7f5d94197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d937fe640 1 -- 172.21.15.49:0/770066790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d94102400 msgr2=0x7f5d94197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d937fe640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d94102400 0x7f5d94197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d937fe640 1 -- 172.21.15.49:0/770066790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d94104d30 msgr2=0x7f5d94198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d937fe640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d94104d30 0x7f5d94198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d937fe640 1 -- 172.21.15.49:0/770066790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9419f7f0 con 0x7f5d94107660 2024-03-26T18:01:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.076+0000 7f5d937fe640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d94107660 0x7f5d9419f180 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c037040 tx=0x7f5d8c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.077+0000 7f5d7bfff640 1 -- 172.21.15.49:0/770066790 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c041020 con 0x7f5d94107660 2024-03-26T18:01:31.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.077+0000 7f5d997df640 1 -- 172.21.15.49:0/770066790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9419fa80 con 0x7f5d94107660 2024-03-26T18:01:31.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.077+0000 7f5d7bfff640 1 -- 172.21.15.49:0/770066790 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d8c034cd0 con 0x7f5d94107660 2024-03-26T18:01:31.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.077+0000 7f5d7bfff640 1 -- 172.21.15.49:0/770066790 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c039040 con 0x7f5d94107660 2024-03-26T18:01:31.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.077+0000 7f5d997df640 1 -- 172.21.15.49:0/770066790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9419ffc0 con 0x7f5d94107660 2024-03-26T18:01:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.079+0000 7f5d7bfff640 1 -- 172.21.15.49:0/770066790 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f5d8c044020 con 0x7f5d94107660 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.079+0000 7f5d9c3c9740 1 Processor -- start 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.079+0000 7f5d9c3c9740 1 -- start start 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.079+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc158060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.079+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc159290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.079+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 0x55f5bc160060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.079+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc0bc270 con 0x55f5bc158e10 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.079+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc064c80 con 0x55f5bc157c40 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.079+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc0b8220 con 0x55f5bc1597d0 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d92ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d92ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60086/0 (socket says 172.21.15.49:60086) 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d92ffd640 1 -- 172.21.15.49:0/2142878050 learned_addr learned my addr 172.21.15.49:0/2142878050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d927fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d927fc640 1 --2- 172.21.15.49:0/2142878050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 0x55f5bc160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d7bfff640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d7c0a2f20 0x7f5d7c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d92ffd640 1 -- 172.21.15.49:0/2142878050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 msgr2=0x55f5bc160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/2142878050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 0x55f5bc160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d92ffd640 1 -- 172.21.15.49:0/2142878050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 msgr2=0x55f5bc158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/2142878050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d92ffd640 1 -- 172.21.15.49:0/2142878050 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5bc076fb0 con 0x55f5bc158e10 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d927fc640 1 --2- 172.21.15.49:0/2142878050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 0x55f5bc160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d937fe640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d7c0a2f20 0x7f5d7c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/2142878050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc159290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009cf0 tx=0x7f5d84011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89e5d6147925b30 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d7bfff640 1 -- 172.21.15.49:0/770066790 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f5d8c0f1560 con 0x7f5d94107660 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d797fa640 1 -- 172.21.15.49:0/2142878050 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8401e070 con 0x55f5bc158e10 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d797fa640 1 -- 172.21.15.49:0/2142878050 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d84019b10 con 0x55f5bc158e10 2024-03-26T18:01:31.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.080+0000 7f5d797fa640 1 -- 172.21.15.49:0/2142878050 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400ea60 con 0x55f5bc158e10 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2142878050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 msgr2=0x55f5bc159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d937fe640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d7c0a2f20 0x7f5d7c0a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0bd970 tx=0x7f5d8c090710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/2142878050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc159290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009cf0 tx=0x7f5d84011810 comp rx=0 tx=0).stop 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2142878050 shutdown_connections 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/2142878050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 0x55f5bc160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/2142878050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc159290 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/2142878050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2142878050 >> 172.21.15.49:0/2142878050 conn(0x55f5bc153030 msgr2=0x55f5bc153430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2142878050 shutdown_connections 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2142878050 wait complete. 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 -- start start 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.081+0000 7f5d9c3c9740 1 Processor -- start 2024-03-26T18:01:31.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc1621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc158e10 0x55f5bc162720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc169760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc1607e0 con 0x55f5bc1597d0 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc077fc0 con 0x55f5bc157c40 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc160960 con 0x55f5bc158e10 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d9c3c9740 10 client.0 _ll_register_callbacks cb 0x55f5bc072770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d927fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc1621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d927fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc169760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d927fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc169760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60100/0 (socket says 172.21.15.49:60100) 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d927fc640 1 -- 172.21.15.49:0/2038470116 learned_addr learned my addr 172.21.15.49:0/2038470116 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc158e10 0x55f5bc162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d927fc640 1 -- 172.21.15.49:0/2038470116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc158e10 msgr2=0x55f5bc162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d927fc640 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc158e10 0x55f5bc162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d927fc640 1 -- 172.21.15.49:0/2038470116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 msgr2=0x55f5bc1621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d927fc640 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc1621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d927fc640 1 -- 172.21.15.49:0/2038470116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5bbdd02d0 con 0x55f5bc1597d0 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d927fc640 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc169760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5d88011b00 tx=0x7f5d880096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d8c5304593ef68 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d76ff5640 1 -- 172.21.15.49:0/2038470116 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88004260 con 0x55f5bc1597d0 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d76ff5640 1 -- 172.21.15.49:0/2038470116 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f5d88004400 con 0x55f5bc1597d0 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.082+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2038470116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d84019b10 con 0x55f5bc1597d0 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d76ff5640 1 -- 172.21.15.49:0/2038470116 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d88029050 con 0x55f5bc1597d0 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d9c3c9740 10 client.15210 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2038470116 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5bbe9a6b0 con 0x55f5bc1597d0 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d76ff5640 1 -- 172.21.15.49:0/2038470116 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d84019b10 con 0x55f5bc1597d0 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d76ff5640 1 -- 172.21.15.49:0/2038470116 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5d88003b80 con 0x55f5bc1597d0 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d9c3c9740 10 client.15210 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:31.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d9c3c9740 10 client.15210 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d9c3c9740 20 client.15210 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x55f5bc16f010 0x55f5bc170480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d9c3c9740 4 client.15210 mds_command: new command op to 24388 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2038470116 --> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5bc0c97b0 con 0x55f5bc16f010 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.083+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x55f5bc16f010 0x55f5bc170480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.084+0000 7f5d76ff5640 10 client.15210 ms_handle_connect on v2:172.21.15.171:6832/2067087026 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.084+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x55f5bc16f010 0x55f5bc170480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b88aa4f24a7e9015 server_cookie=2032e119d7e5c232 in_seq=0 out_seq=0 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.084+0000 7f5d76ff5640 1 -- 172.21.15.49:0/2038470116 <== mds.4 v2:172.21.15.171:6832/2067087026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f5bc0c97b0 con 0x55f5bc16f010 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.084+0000 7f5d76ff5640 10 client.15210 handle_command_reply: tid=0 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.084+0000 7f5d9c3c9740 1 client.15210 shutdown 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.084+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2038470116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 msgr2=0x55f5bc169760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.084+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc169760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5d88011b00 tx=0x7f5d880096c0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.085+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2038470116 shutdown_connections 2024-03-26T18:01:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.085+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x55f5bc16f010 0x55f5bc170480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.085+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc158e10 0x55f5bc162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.085+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc169760 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.085+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/2038470116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc1621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.085+0000 7f5d76ff5640 0 client.15210 ms_handle_reset on v2:172.21.15.171:6832/2067087026 2024-03-26T18:01:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.085+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2038470116 >> 172.21.15.49:0/2038470116 conn(0x55f5bc153030 msgr2=0x55f5bc152230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.085+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2038470116 shutdown_connections 2024-03-26T18:01:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.085+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/2038470116 wait complete. 2024-03-26T18:01:31.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.085+0000 7f5d9c3c9740 20 client.15210 trim_cache size 0 max 16384 2024-03-26T18:01:31.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d9c3c9740 1 Processor -- start 2024-03-26T18:01:31.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d9c3c9740 1 -- start start 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc157c40 0x55f5bc160d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc158e10 0x55f5bc15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc16e7c0 con 0x55f5bc1597d0 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc16ee90 con 0x55f5bc158e10 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc16f560 con 0x55f5bc157c40 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d927fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d927fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc15cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60112/0 (socket says 172.21.15.49:60112) 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d927fc640 1 -- 172.21.15.49:0/491931672 learned_addr learned my addr 172.21.15.49:0/491931672 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/491931672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc158e10 0x55f5bc15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d927fc640 1 --2- 172.21.15.49:0/491931672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc157c40 0x55f5bc160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d927fc640 1 -- 172.21.15.49:0/491931672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc157c40 msgr2=0x55f5bc160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d927fc640 1 --2- 172.21.15.49:0/491931672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc157c40 0x55f5bc160d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d927fc640 1 -- 172.21.15.49:0/491931672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc158e10 msgr2=0x55f5bc15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d927fc640 1 --2- 172.21.15.49:0/491931672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc158e10 0x55f5bc15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d927fc640 1 -- 172.21.15.49:0/491931672 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5bc157840 con 0x55f5bc1597d0 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/491931672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc158e10 0x55f5bc15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d927fc640 1 --2- 172.21.15.49:0/491931672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc15cf30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5d880119f0 tx=0x7f5d8800ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c32cd68c5e7777 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.103+0000 7f5d78ff9640 1 -- 172.21.15.49:0/491931672 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88026040 con 0x55f5bc1597d0 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d78ff9640 1 -- 172.21.15.49:0/491931672 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d88020640 con 0x55f5bc1597d0 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d78ff9640 1 -- 172.21.15.49:0/491931672 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88005290 con 0x55f5bc1597d0 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/491931672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 msgr2=0x55f5bc15cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/491931672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc15cf30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5d880119f0 tx=0x7f5d8800ffd0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/491931672 shutdown_connections 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/491931672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc1597d0 0x55f5bc15cf30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/491931672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc158e10 0x55f5bc15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/491931672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc157c40 0x55f5bc160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/491931672 >> 172.21.15.49:0/491931672 conn(0x55f5bc153030 msgr2=0x55f5bc153410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/491931672 shutdown_connections 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/491931672 wait complete. 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 -- start start 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.104+0000 7f5d9c3c9740 1 Processor -- start 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc1d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc1d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d9c3c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 0x55f5bc1d7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc170300 con 0x55f5bc158e10 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc16fc30 con 0x55f5bc157c40 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d9c3c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f5bc1709d0 con 0x55f5bc1597d0 2024-03-26T18:01:31.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d9c3c9740 10 client.0 _ll_register_callbacks cb 0x7f5d88009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d92ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc1d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d927fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc1d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d92ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc1d52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60116/0 (socket says 172.21.15.49:60116) 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d92ffd640 1 -- 172.21.15.49:0/1640454361 learned_addr learned my addr 172.21.15.49:0/1640454361 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d927fc640 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 0x55f5bc1d7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d92ffd640 1 -- 172.21.15.49:0/1640454361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 msgr2=0x55f5bc1d7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 0x55f5bc1d7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d92ffd640 1 -- 172.21.15.49:0/1640454361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 msgr2=0x55f5bc1d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc1d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d92ffd640 1 -- 172.21.15.49:0/1640454361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5bc1d7d50 con 0x55f5bc158e10 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d92ffd640 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc1d52c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5d8401c040 tx=0x7f5d84009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e9b02facb6e9ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d76ff5640 1 -- 172.21.15.49:0/1640454361 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8402d070 con 0x55f5bc158e10 2024-03-26T18:01:31.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d76ff5640 1 -- 172.21.15.49:0/1640454361 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f5d84005240 con 0x55f5bc158e10 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.105+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/1640454361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d88020640 con 0x55f5bc158e10 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d76ff5640 1 -- 172.21.15.49:0/1640454361 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d84036050 con 0x55f5bc158e10 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d9c3c9740 10 client.15216 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/1640454361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5bc1d90c0 con 0x55f5bc158e10 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d76ff5640 1 -- 172.21.15.49:0/1640454361 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d88020640 con 0x55f5bc158e10 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d76ff5640 1 -- 172.21.15.49:0/1640454361 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5d84030020 con 0x55f5bc158e10 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d9c3c9740 10 client.15216 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d9c3c9740 10 client.15216 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d9c3c9740 20 client.15216 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x55f5bc1ddaa0 0x55f5bc1dc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d9c3c9740 4 client.15216 mds_command: new command op to 24388 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/1640454361 --> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] -- command(tid 0: {"prefix": "status"}) -- 0x55f5bc0c3b00 con 0x55f5bc1ddaa0 2024-03-26T18:01:31.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d927fc640 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x55f5bc1ddaa0 0x55f5bc1dc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d927fc640 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x55f5bc1ddaa0 0x55f5bc1dc7b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2440bed31c5dca7a server_cookie=eca47a7c123ca0b7 in_seq=0 out_seq=0 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.106+0000 7f5d76ff5640 10 client.15216 ms_handle_connect on v2:172.21.15.171:6832/2067087026 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d76ff5640 1 -- 172.21.15.49:0/1640454361 <== mds.4 v2:172.21.15.171:6832/2067087026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f5bc0c3b00 con 0x55f5bc1ddaa0 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d76ff5640 10 client.15216 handle_command_reply: tid=0 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 client.15216 shutdown 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/1640454361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 msgr2=0x55f5bc1d52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc1d52c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5d8401c040 tx=0x7f5d84009990 comp rx=0 tx=0).stop 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/1640454361 shutdown_connections 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x55f5bc1ddaa0 0x55f5bc1dc7b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f5bc1597d0 0x55f5bc1d7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d76ff5640 0 client.15216 ms_handle_reset on v2:172.21.15.171:6832/2067087026 2024-03-26T18:01:31.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f5bc158e10 0x55f5bc1d52c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 --2- 172.21.15.49:0/1640454361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55f5bc157c40 0x55f5bc1d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/1640454361 >> 172.21.15.49:0/1640454361 conn(0x55f5bc153030 msgr2=0x55f5bc15c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/1640454361 shutdown_connections 2024-03-26T18:01:31.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 1 -- 172.21.15.49:0/1640454361 wait complete. 2024-03-26T18:01:31.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d9c3c9740 20 client.15216 trim_cache size 0 max 16384 2024-03-26T18:01:31.098 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:01:31.098 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T18:01:31.098 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 4, 2024-03-26T18:01:31.098 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24388, 2024-03-26T18:01:31.098 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T18:01:31.098 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T18:01:31.098 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T18:01:31.098 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 82.661065593000004, 2024-03-26T18:01:31.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2024-03-26T18:01:31.099 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 92, 2024-03-26T18:01:31.099 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T18:01:31.099 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 139.252430073 2024-03-26T18:01:31.099 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:01:31.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d997df640 1 -- 172.21.15.49:0/770066790 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d7c0a2f20 msgr2=0x7f5d7c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d997df640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d7c0a2f20 0x7f5d7c0a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0bd970 tx=0x7f5d8c090710 comp rx=0 tx=0).stop 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d997df640 1 -- 172.21.15.49:0/770066790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d94107660 msgr2=0x7f5d9419f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.107+0000 7f5d997df640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d94107660 0x7f5d9419f180 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c037040 tx=0x7f5d8c032f40 comp rx=0 tx=0).stop 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.108+0000 7f5d997df640 1 -- 172.21.15.49:0/770066790 shutdown_connections 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.108+0000 7f5d997df640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5d7c0a2f20 0x7f5d7c0a53e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.108+0000 7f5d997df640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d94107660 0x7f5d9419f180 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.108+0000 7f5d997df640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d94104d30 0x7f5d94198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.108+0000 7f5d997df640 1 --2- 172.21.15.49:0/770066790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d94102400 0x7f5d94197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.108+0000 7f5d997df640 1 -- 172.21.15.49:0/770066790 >> 172.21.15.49:0/770066790 conn(0x7f5d940fbd70 msgr2=0x7f5d94102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.108+0000 7f5d997df640 1 -- 172.21.15.49:0/770066790 shutdown_connections 2024-03-26T18:01:31.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.108+0000 7f5d997df640 1 -- 172.21.15.49:0/770066790 wait complete. 2024-03-26T18:01:31.102 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-26T18:01:31.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.229+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1823706862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8241019b0 msgr2=0x7ff824101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.229+0000 7ff82a4c6640 1 --2- 172.21.15.49:0/1823706862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8241019b0 0x7ff824101e30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff818009900 tx=0x7ff81802bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:31.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.229+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1823706862 shutdown_connections 2024-03-26T18:01:31.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.229+0000 7ff82a4c6640 1 --2- 172.21.15.49:0/1823706862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff824102370 0x7ff824108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.229+0000 7ff82a4c6640 1 --2- 172.21.15.49:0/1823706862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8241019b0 0x7ff824101e30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.229+0000 7ff82a4c6640 1 --2- 172.21.15.49:0/1823706862 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8241007c0 0x7ff824100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.229+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1823706862 >> 172.21.15.49:0/1823706862 conn(0x7ff8240fbd50 msgr2=0x7ff8240fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.229+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1823706862 shutdown_connections 2024-03-26T18:01:31.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.229+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1823706862 wait complete. 2024-03-26T18:01:31.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff82a4c6640 1 Processor -- start 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff82a4c6640 1 -- start start 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff82a4c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8241007c0 0x7ff824197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff82a4c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8241019b0 0x7ff824198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff82a4c6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff824102370 0x7ff82419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff82a4c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff82410c470 con 0x7ff8241007c0 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff82a4c6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ff82410c2f0 con 0x7ff8241019b0 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff82a4c6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff82410c5f0 con 0x7ff824102370 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff823fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8241007c0 0x7ff824197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff823fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8241007c0 0x7ff824197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60128/0 (socket says 172.21.15.49:60128) 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff823fff640 1 -- 172.21.15.49:0/1840214213 learned_addr learned my addr 172.21.15.49:0/1840214213 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff8237fe640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8241019b0 0x7ff824198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.230+0000 7ff828a63640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff824102370 0x7ff82419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff823fff640 1 -- 172.21.15.49:0/1840214213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff824102370 msgr2=0x7ff82419f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff823fff640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff824102370 0x7ff82419f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff823fff640 1 -- 172.21.15.49:0/1840214213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8241019b0 msgr2=0x7ff824198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff823fff640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8241019b0 0x7ff824198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff823fff640 1 -- 172.21.15.49:0/1840214213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82419f880 con 0x7ff8241007c0 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff8237fe640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8241019b0 0x7ff824198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff823fff640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8241007c0 0x7ff824197ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e9c0 tx=0x7ff81400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff8217fa640 1 -- 172.21.15.49:0/1840214213 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400ccc0 con 0x7ff8241007c0 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff8217fa640 1 -- 172.21.15.49:0/1840214213 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8140035f0 con 0x7ff8241007c0 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff8217fa640 1 -- 172.21.15.49:0/1840214213 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400ce60 con 0x7ff8241007c0 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1840214213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82419fb70 con 0x7ff8241007c0 2024-03-26T18:01:31.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.231+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1840214213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241a0110 con 0x7ff8241007c0 2024-03-26T18:01:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.234+0000 7ff8217fa640 1 -- 172.21.15.49:0/1840214213 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7ff8140037a0 con 0x7ff8241007c0 2024-03-26T18:01:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.234+0000 7ff82d18e740 1 Processor -- start 2024-03-26T18:01:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.234+0000 7ff82d18e740 1 -- start start 2024-03-26T18:01:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.234+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9658060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.234+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab9659290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.234+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab9660060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff82d18e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eab95bc270 con 0x55eab9658e10 2024-03-26T18:01:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff82d18e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55eab9564c80 con 0x55eab9657c40 2024-03-26T18:01:31.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff82d18e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55eab95b8220 con 0x55eab96597d0 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff823fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab9659290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff828a63640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9658060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff823fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab9659290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60140/0 (socket says 172.21.15.49:60140) 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff828a63640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9658060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:38340/0 (socket says 172.21.15.49:38340) 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff823fff640 1 -- 172.21.15.49:0/1419396218 learned_addr learned my addr 172.21.15.49:0/1419396218 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab9660060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff8217fa640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff80c0a2fa0 0x7ff80c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff8237fe640 1 -- 172.21.15.49:0/1419396218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 msgr2=0x55eab9658060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff8237fe640 1 --2- 172.21.15.49:0/1419396218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9658060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff8237fe640 1 -- 172.21.15.49:0/1419396218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 msgr2=0x55eab9659290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff8237fe640 1 --2- 172.21.15.49:0/1419396218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab9659290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff8237fe640 1 -- 172.21.15.49:0/1419396218 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eab9576fb0 con 0x55eab96597d0 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff8217fa640 1 -- 172.21.15.49:0/1840214213 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7ff8140cfa60 con 0x7ff8241007c0 2024-03-26T18:01:31.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff828a63640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff80c0a2fa0 0x7ff80c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.235+0000 7ff8237fe640 1 --2- 172.21.15.49:0/1419396218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab9660060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff818036040 tx=0x7ff818009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73fb83f13b3a7786 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff80a7fc640 1 -- 172.21.15.49:0/1419396218 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81802cd30 con 0x55eab96597d0 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff828a63640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff80c0a2fa0 0x7ff80c0a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55eab9658d10 tx=0x7ff81c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff80a7fc640 1 -- 172.21.15.49:0/1419396218 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8180046c0 con 0x55eab96597d0 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff80a7fc640 1 -- 172.21.15.49:0/1419396218 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818041ae0 con 0x55eab96597d0 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff82d18e740 1 -- 172.21.15.49:0/1419396218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 msgr2=0x55eab9660060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff82d18e740 1 --2- 172.21.15.49:0/1419396218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab9660060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff818036040 tx=0x7ff818009560 comp rx=0 tx=0).stop 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff82d18e740 1 -- 172.21.15.49:0/1419396218 shutdown_connections 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff82d18e740 1 --2- 172.21.15.49:0/1419396218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab9660060 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff82d18e740 1 --2- 172.21.15.49:0/1419396218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab9659290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff82d18e740 1 --2- 172.21.15.49:0/1419396218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9658060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff82d18e740 1 -- 172.21.15.49:0/1419396218 >> 172.21.15.49:0/1419396218 conn(0x55eab9653030 msgr2=0x55eab9653430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.236+0000 7ff82d18e740 1 -- 172.21.15.49:0/1419396218 shutdown_connections 2024-03-26T18:01:31.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.237+0000 7ff82d18e740 1 -- 172.21.15.49:0/1419396218 wait complete. 2024-03-26T18:01:31.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.237+0000 7ff82d18e740 1 -- start start 2024-03-26T18:01:31.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.237+0000 7ff82d18e740 1 Processor -- start 2024-03-26T18:01:31.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.237+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9662190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.237+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab96626d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.237+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab9669710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.237+0000 7ff82d18e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eab9660780 con 0x55eab9658e10 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.237+0000 7ff82d18e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55eab9577fc0 con 0x55eab9657c40 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.237+0000 7ff82d18e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55eab9660900 con 0x55eab96597d0 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.237+0000 7ff82d18e740 10 client.0 _ll_register_callbacks cb 0x55eab9572770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff828a63640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab9669710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff823fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9662190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab96626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff823fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9662190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:38348/0 (socket says 172.21.15.49:38348) 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab96626d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60156/0 (socket says 172.21.15.49:60156) 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff823fff640 1 -- 172.21.15.49:0/98412971 learned_addr learned my addr 172.21.15.49:0/98412971 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8237fe640 1 -- 172.21.15.49:0/98412971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 msgr2=0x55eab9669710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8237fe640 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab9669710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8237fe640 1 -- 172.21.15.49:0/98412971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 msgr2=0x55eab9662190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8237fe640 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9662190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8237fe640 1 -- 172.21.15.49:0/98412971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eab92d02d0 con 0x55eab9658e10 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff823fff640 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9662190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8237fe640 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab96626d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff81802c8e0 tx=0x7ff818005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfa1984315a9c90 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8037fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8037fe640 1 -- 172.21.15.49:0/98412971 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818053070 con 0x55eab9658e10 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff8037fe640 1 -- 172.21.15.49:0/98412971 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7ff818041c30 con 0x55eab9658e10 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.238+0000 7ff82d18e740 1 -- 172.21.15.49:0/98412971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8180046c0 con 0x55eab9658e10 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff8037fe640 1 -- 172.21.15.49:0/98412971 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff81805f050 con 0x55eab9658e10 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff82d18e740 10 client.15228 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff82d18e740 1 -- 172.21.15.49:0/98412971 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eab939a6b0 con 0x55eab9658e10 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff8037fe640 1 -- 172.21.15.49:0/98412971 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8180046c0 con 0x55eab9658e10 2024-03-26T18:01:31.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff8037fe640 1 -- 172.21.15.49:0/98412971 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff81804c780 con 0x55eab9658e10 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff82d18e740 10 client.15228 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff82d18e740 10 client.15228 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff82d18e740 20 client.15228 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff82d18e740 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x55eab966ea50 0x55eab966fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff82d18e740 4 client.15228 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff82d18e740 1 -- 172.21.15.49:0/98412971 --> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eab95c97b0 con 0x55eab966ea50 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.239+0000 7ff823fff640 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x55eab966ea50 0x55eab966fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.240+0000 7ff823fff640 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x55eab966ea50 0x55eab966fec0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bcf8dc91f8d24de server_cookie=dd95a2d19f95fa7c in_seq=0 out_seq=0 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.240+0000 7ff8037fe640 10 client.15228 ms_handle_connect on v2:172.21.15.115:6832/3604276420 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff8037fe640 1 -- 172.21.15.49:0/98412971 <== mds.3 v2:172.21.15.115:6832/3604276420 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55eab95c97b0 con 0x55eab966ea50 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff8037fe640 10 client.15228 handle_command_reply: tid=0 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 client.15228 shutdown 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 -- 172.21.15.49:0/98412971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 msgr2=0x55eab96626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab96626d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff81802c8e0 tx=0x7ff818005dd0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 -- 172.21.15.49:0/98412971 shutdown_connections 2024-03-26T18:01:31.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x55eab966ea50 0x55eab966fec0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab9669710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9658e10 0x55eab96626d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 --2- 172.21.15.49:0/98412971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9657c40 0x55eab9662190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff8037fe640 0 client.15228 ms_handle_reset on v2:172.21.15.115:6832/3604276420 2024-03-26T18:01:31.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 -- 172.21.15.49:0/98412971 >> 172.21.15.49:0/98412971 conn(0x55eab9653030 msgr2=0x55eab9652230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 -- 172.21.15.49:0/98412971 shutdown_connections 2024-03-26T18:01:31.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 1 -- 172.21.15.49:0/98412971 wait complete. 2024-03-26T18:01:31.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.241+0000 7ff82d18e740 20 client.15228 trim_cache size 0 max 16384 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.259+0000 7ff82d18e740 1 Processor -- start 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.259+0000 7ff82d18e740 1 -- start start 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab9657c40 0x55eab966dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 0x55eab96696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab96597d0 0x55eab965c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff82d18e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eab966e200 con 0x55eab96597d0 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff82d18e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55eab966e8d0 con 0x55eab9658e10 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff82d18e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55eab966efa0 con 0x55eab9657c40 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff828a63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab96597d0 0x55eab965c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff828a63640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab96597d0 0x55eab965c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60158/0 (socket says 172.21.15.49:60158) 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff828a63640 1 -- 172.21.15.49:0/1943515951 learned_addr learned my addr 172.21.15.49:0/1943515951 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff823fff640 1 --2- 172.21.15.49:0/1943515951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab9657c40 0x55eab966dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff8237fe640 1 --2- 172.21.15.49:0/1943515951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 0x55eab96696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff828a63640 1 -- 172.21.15.49:0/1943515951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab9657c40 msgr2=0x55eab966dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff828a63640 1 --2- 172.21.15.49:0/1943515951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab9657c40 0x55eab966dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff828a63640 1 -- 172.21.15.49:0/1943515951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 msgr2=0x55eab96696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff828a63640 1 --2- 172.21.15.49:0/1943515951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 0x55eab96696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff828a63640 1 -- 172.21.15.49:0/1943515951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eab9663030 con 0x55eab96597d0 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff828a63640 1 --2- 172.21.15.49:0/1943515951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab96597d0 0x55eab965c040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009b80 tx=0x7ff81c009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5421412f55f98320 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff809ffb640 1 -- 172.21.15.49:0/1943515951 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004b20 con 0x55eab96597d0 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff809ffb640 1 -- 172.21.15.49:0/1943515951 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c004cc0 con 0x55eab96597d0 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.260+0000 7ff809ffb640 1 -- 172.21.15.49:0/1943515951 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c005770 con 0x55eab96597d0 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.261+0000 7ff82d18e740 1 -- 172.21.15.49:0/1943515951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab96597d0 msgr2=0x55eab965c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.261+0000 7ff82d18e740 1 --2- 172.21.15.49:0/1943515951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab96597d0 0x55eab965c040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009b80 tx=0x7ff81c009b10 comp rx=0 tx=0).stop 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.261+0000 7ff82d18e740 1 -- 172.21.15.49:0/1943515951 shutdown_connections 2024-03-26T18:01:31.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.261+0000 7ff82d18e740 1 --2- 172.21.15.49:0/1943515951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab96597d0 0x55eab965c040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.261+0000 7ff82d18e740 1 --2- 172.21.15.49:0/1943515951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 0x55eab96696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.261+0000 7ff82d18e740 1 --2- 172.21.15.49:0/1943515951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab9657c40 0x55eab966dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.261+0000 7ff82d18e740 1 -- 172.21.15.49:0/1943515951 >> 172.21.15.49:0/1943515951 conn(0x55eab9653030 msgr2=0x55eab9660d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.261+0000 7ff82d18e740 1 -- 172.21.15.49:0/1943515951 shutdown_connections 2024-03-26T18:01:31.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.261+0000 7ff82d18e740 1 -- 172.21.15.49:0/1943515951 wait complete. 2024-03-26T18:01:31.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.261+0000 7ff82d18e740 1 -- start start 2024-03-26T18:01:31.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff82d18e740 1 Processor -- start 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9657c40 0x55eab96654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 0x55eab9665a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff82d18e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab96689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff82d18e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55eab966fd40 con 0x55eab9657c40 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff82d18e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55eab966f670 con 0x55eab9658e10 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff82d18e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55eab9670410 con 0x55eab96597d0 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff82d18e740 10 client.0 _ll_register_callbacks cb 0x7ff8140980e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff823fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9657c40 0x55eab96654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 0x55eab9665a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff8237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 0x55eab9665a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:38374/0 (socket says 172.21.15.49:38374) 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff8237fe640 1 -- 172.21.15.49:0/3941342817 learned_addr learned my addr 172.21.15.49:0/3941342817 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff828a63640 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab96689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff823fff640 1 -- 172.21.15.49:0/3941342817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 msgr2=0x55eab96689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff823fff640 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab96689e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff823fff640 1 -- 172.21.15.49:0/3941342817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 msgr2=0x55eab9665a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff823fff640 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 0x55eab9665a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff823fff640 1 -- 172.21.15.49:0/3941342817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eab9658630 con 0x55eab9657c40 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff8237fe640 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 0x55eab9665a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff823fff640 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9657c40 0x55eab96654f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff8140a4330 tx=0x7ff814098190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a47de125f7cff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.262+0000 7ff8037fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff8037fe640 1 -- 172.21.15.49:0/3941342817 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8140b4070 con 0x55eab9657c40 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff8037fe640 1 -- 172.21.15.49:0/3941342817 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7ff814003b30 con 0x55eab9657c40 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff82d18e740 1 -- 172.21.15.49:0/3941342817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81c004cc0 con 0x55eab9657c40 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff8037fe640 1 -- 172.21.15.49:0/3941342817 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff81409b7b0 con 0x55eab9657c40 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff82d18e740 10 client.15234 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:31.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff82d18e740 1 -- 172.21.15.49:0/3941342817 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eab96d4d00 con 0x55eab9657c40 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff8037fe640 1 -- 172.21.15.49:0/3941342817 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c004cc0 con 0x55eab9657c40 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff8037fe640 1 -- 172.21.15.49:0/3941342817 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff81409c0b0 con 0x55eab9657c40 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff82d18e740 10 client.15234 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff82d18e740 10 client.15234 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff82d18e740 20 client.15234 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff82d18e740 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x55eab96d9920 0x55eab96d8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff82d18e740 4 client.15234 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff82d18e740 1 -- 172.21.15.49:0/3941342817 --> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] -- command(tid 0: {"prefix": "status"}) -- 0x55eab95c3b00 con 0x55eab96d9920 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.263+0000 7ff8237fe640 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x55eab96d9920 0x55eab96d8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff8237fe640 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x55eab96d9920 0x55eab96d8630 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac0edc56570eb22f server_cookie=cc5ead82862b5158 in_seq=0 out_seq=0 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff8037fe640 10 client.15234 ms_handle_connect on v2:172.21.15.115:6832/3604276420 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff8037fe640 1 -- 172.21.15.49:0/3941342817 <== mds.3 v2:172.21.15.115:6832/3604276420 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55eab95c3b00 con 0x55eab96d9920 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff8037fe640 10 client.15234 handle_command_reply: tid=0 2024-03-26T18:01:31.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 client.15234 shutdown 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 -- 172.21.15.49:0/3941342817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9657c40 msgr2=0x55eab96654f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9657c40 0x55eab96654f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff8140a4330 tx=0x7ff814098190 comp rx=0 tx=0).stop 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 -- 172.21.15.49:0/3941342817 shutdown_connections 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x55eab96d9920 0x55eab96d8630 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eab96597d0 0x55eab96689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55eab9658e10 0x55eab9665a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff8037fe640 0 client.15234 ms_handle_reset on v2:172.21.15.115:6832/3604276420 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 --2- 172.21.15.49:0/3941342817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eab9657c40 0x55eab96654f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 -- 172.21.15.49:0/3941342817 >> 172.21.15.49:0/3941342817 conn(0x55eab9653030 msgr2=0x55eab9653410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 -- 172.21.15.49:0/3941342817 shutdown_connections 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 1 -- 172.21.15.49:0/3941342817 wait complete. 2024-03-26T18:01:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.264+0000 7ff82d18e740 20 client.15234 trim_cache size 0 max 16384 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 3, 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24428, 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 83.823509487999999, 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 92, 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T18:01:31.253 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 136.594365522 2024-03-26T18:01:31.254 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:01:31.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1840214213 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff80c0a2fa0 msgr2=0x7ff80c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff80c0a2fa0 0x7ff80c0a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55eab9658d10 tx=0x7ff81c004ab0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1840214213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8241007c0 msgr2=0x7ff824197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8241007c0 0x7ff824197ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e9c0 tx=0x7ff81400edc0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1840214213 shutdown_connections 2024-03-26T18:01:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7ff80c0a2fa0 0x7ff80c0a5460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff824102370 0x7ff82419f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8241019b0 0x7ff824198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 --2- 172.21.15.49:0/1840214213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8241007c0 0x7ff824197ce0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1840214213 >> 172.21.15.49:0/1840214213 conn(0x7ff8240fbd50 msgr2=0x7ff8240fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1840214213 shutdown_connections 2024-03-26T18:01:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.265+0000 7ff82a4c6640 1 -- 172.21.15.49:0/1840214213 wait complete. 2024-03-26T18:01:31.257 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-26T18:01:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.386+0000 7f12b5dc4640 1 -- 172.21.15.49:0/2037406019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b0103490 msgr2=0x7f12b0109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.386+0000 7f12b5dc4640 1 --2- 172.21.15.49:0/2037406019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b0103490 0x7f12b0109d20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009ca0 tx=0x7f12a802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.386+0000 7f12b5dc4640 1 -- 172.21.15.49:0/2037406019 shutdown_connections 2024-03-26T18:01:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.386+0000 7f12b5dc4640 1 --2- 172.21.15.49:0/2037406019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b0103490 0x7f12b0109d20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.386+0000 7f12b5dc4640 1 --2- 172.21.15.49:0/2037406019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b0102ad0 0x7f12b0102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.386+0000 7f12b5dc4640 1 --2- 172.21.15.49:0/2037406019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b01018e0 0x7f12b0101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.386+0000 7f12b5dc4640 1 -- 172.21.15.49:0/2037406019 >> 172.21.15.49:0/2037406019 conn(0x7f12b00fce50 msgr2=0x7f12b00ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.386+0000 7f12b5dc4640 1 -- 172.21.15.49:0/2037406019 shutdown_connections 2024-03-26T18:01:31.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.386+0000 7f12b5dc4640 1 -- 172.21.15.49:0/2037406019 wait complete. 2024-03-26T18:01:31.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12b5dc4640 1 Processor -- start 2024-03-26T18:01:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12b5dc4640 1 -- start start 2024-03-26T18:01:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12b5dc4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b01018e0 0x7f12b0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12b5dc4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b0102ad0 0x7f12b0198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12b5dc4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b0103490 0x7f12b019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12b5dc4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12b010d560 con 0x7f12b0102ad0 2024-03-26T18:01:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12b5dc4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f12b010d3e0 con 0x7f12b0103490 2024-03-26T18:01:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12b5dc4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f12b010d6e0 con 0x7f12b01018e0 2024-03-26T18:01:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b0102ad0 0x7f12b0198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b0102ad0 0x7f12b0198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60192/0 (socket says 172.21.15.49:60192) 2024-03-26T18:01:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12aeffd640 1 -- 172.21.15.49:0/299268456 learned_addr learned my addr 172.21.15.49:0/299268456 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12affff640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b0103490 0x7f12b019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.387+0000 7f12af7fe640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b01018e0 0x7f12b0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12aeffd640 1 -- 172.21.15.49:0/299268456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b01018e0 msgr2=0x7f12b0197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12aeffd640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b01018e0 0x7f12b0197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12aeffd640 1 -- 172.21.15.49:0/299268456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b0103490 msgr2=0x7f12b019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12aeffd640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b0103490 0x7f12b019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12aeffd640 1 -- 172.21.15.49:0/299268456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b019f830 con 0x7f12b0102ad0 2024-03-26T18:01:31.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12aeffd640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b0102ad0 0x7f12b0198160 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f12a400e9f0 tx=0x7f12a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12acff9640 1 -- 172.21.15.49:0/299268456 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a400cd80 con 0x7f12b0102ad0 2024-03-26T18:01:31.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12acff9640 1 -- 172.21.15.49:0/299268456 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a4010910 con 0x7f12b0102ad0 2024-03-26T18:01:31.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12b5dc4640 1 -- 172.21.15.49:0/299268456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b019fb20 con 0x7f12b0102ad0 2024-03-26T18:01:31.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12acff9640 1 -- 172.21.15.49:0/299268456 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4018990 con 0x7f12b0102ad0 2024-03-26T18:01:31.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.388+0000 7f12b5dc4640 1 -- 172.21.15.49:0/299268456 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01a0070 con 0x7f12b0102ad0 2024-03-26T18:01:31.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.390+0000 7f12acff9640 1 -- 172.21.15.49:0/299268456 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f12a4010ac0 con 0x7f12b0102ad0 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.390+0000 7f12b89ab740 1 Processor -- start 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.390+0000 7f12b89ab740 1 -- start start 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b13c40 0x556966b14060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b15290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b157d0 0x556966b1c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12b89ab740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556966a78270 con 0x556966b13c40 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12b89ab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556966a20c80 con 0x556966b14e10 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12b89ab740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556966a74220 con 0x556966b157d0 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b13c40 0x556966b14060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b13c40 0x556966b14060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60202/0 (socket says 172.21.15.49:60202) 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12affff640 1 -- 172.21.15.49:0/1962262726 learned_addr learned my addr 172.21.15.49:0/1962262726 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12acff9640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12980a2f20 0x7f12980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12af7fe640 1 --2- 172.21.15.49:0/1962262726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b15290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12aeffd640 1 --2- 172.21.15.49:0/1962262726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b157d0 0x556966b1c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12affff640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12980a2f20 0x7f12980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12af7fe640 1 -- 172.21.15.49:0/1962262726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b157d0 msgr2=0x556966b1c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12af7fe640 1 --2- 172.21.15.49:0/1962262726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b157d0 0x556966b1c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12acff9640 1 -- 172.21.15.49:0/299268456 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f12a40cfab0 con 0x7f12b0102ad0 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12af7fe640 1 -- 172.21.15.49:0/1962262726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b13c40 msgr2=0x556966b14060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12af7fe640 1 --2- 172.21.15.49:0/1962262726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b13c40 0x556966b14060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12af7fe640 1 -- 172.21.15.49:0/1962262726 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556966a32fb0 con 0x556966b14e10 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12affff640 1 --2- 172.21.15.49:0/1962262726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b13c40 0x556966b14060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.391+0000 7f12af7fe640 1 --2- 172.21.15.49:0/1962262726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b15290 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12b0102930 tx=0x7f12a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bcf1583e1e01884 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f1295ffb640 1 -- 172.21.15.49:0/1962262726 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0004220 con 0x556966b14e10 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f12affff640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12980a2f20 0x7f12980a53e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x556966b14d10 tx=0x7f12a802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f1295ffb640 1 -- 172.21.15.49:0/1962262726 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a0009e10 con 0x556966b14e10 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f1295ffb640 1 -- 172.21.15.49:0/1962262726 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0019790 con 0x556966b14e10 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f12b89ab740 1 -- 172.21.15.49:0/1962262726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 msgr2=0x556966b15290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f12b89ab740 1 --2- 172.21.15.49:0/1962262726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b15290 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12b0102930 tx=0x7f12a0011810 comp rx=0 tx=0).stop 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f12b89ab740 1 -- 172.21.15.49:0/1962262726 shutdown_connections 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f12b89ab740 1 --2- 172.21.15.49:0/1962262726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b157d0 0x556966b1c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f12b89ab740 1 --2- 172.21.15.49:0/1962262726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b15290 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f12b89ab740 1 --2- 172.21.15.49:0/1962262726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b13c40 0x556966b14060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f12b89ab740 1 -- 172.21.15.49:0/1962262726 >> 172.21.15.49:0/1962262726 conn(0x556966b0f030 msgr2=0x556966b0f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f12b89ab740 1 -- 172.21.15.49:0/1962262726 shutdown_connections 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.392+0000 7f12b89ab740 1 -- 172.21.15.49:0/1962262726 wait complete. 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12b89ab740 1 -- start start 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12b89ab740 1 Processor -- start 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 0x556966b1e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b1e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 0x556966b256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12b89ab740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556966b1c7e0 con 0x556966b157d0 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12b89ab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556966a33fc0 con 0x556966b14e10 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12b89ab740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556966b1c960 con 0x556966b13c40 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12b89ab740 10 client.0 _ll_register_callbacks cb 0x556966a2e770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b1e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 0x556966b256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b1e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:38422/0 (socket says 172.21.15.49:38422) 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12aeffd640 1 -- 172.21.15.49:0/3815868584 learned_addr learned my addr 172.21.15.49:0/3815868584 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.393+0000 7f12af7fe640 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 0x556966b1e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12aeffd640 1 -- 172.21.15.49:0/3815868584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 msgr2=0x556966b1e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12aeffd640 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 0x556966b1e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12aeffd640 1 -- 172.21.15.49:0/3815868584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 msgr2=0x556966b256d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12aeffd640 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 0x556966b256d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12aeffd640 1 -- 172.21.15.49:0/3815868584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696678c2d0 con 0x556966b14e10 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12aeffd640 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b1e690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f12a4098320 tx=0x7f12a409a090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=357699fd2515b445 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f128effd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f128effd640 1 -- 172.21.15.49:0/3815868584 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4026040 con 0x556966b14e10 2024-03-26T18:01:31.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f128effd640 1 -- 172.21.15.49:0/3815868584 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f12a409b5b0 con 0x556966b14e10 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12b89ab740 1 -- 172.21.15.49:0/3815868584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a0009e10 con 0x556966b14e10 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f128effd640 1 -- 172.21.15.49:0/3815868584 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f12a409fff0 con 0x556966b14e10 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12b89ab740 10 client.25142 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12b89ab740 1 -- 172.21.15.49:0/3815868584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569668566b0 con 0x556966b14e10 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f128effd640 1 -- 172.21.15.49:0/3815868584 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a0009e10 con 0x556966b14e10 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f128effd640 1 -- 172.21.15.49:0/3815868584 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f12a40b8020 con 0x556966b14e10 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12b89ab740 10 client.25142 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12b89ab740 10 client.25142 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12b89ab740 20 client.25142 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12b89ab740 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x556966b2ae90 0x556966b2c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12b89ab740 4 client.25142 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:31.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.394+0000 7f12b89ab740 1 -- 172.21.15.49:0/3815868584 --> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556966a857b0 con 0x556966b2ae90 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.395+0000 7f12af7fe640 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x556966b2ae90 0x556966b2c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.395+0000 7f128effd640 10 client.25142 ms_handle_connect on v2:172.21.15.171:6838/1082475840 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.395+0000 7f12af7fe640 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x556966b2ae90 0x556966b2c300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbda48a72049d6dd server_cookie=2b8c23d94cdb17b1 in_seq=0 out_seq=0 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f128effd640 1 -- 172.21.15.49:0/3815868584 <== mds.2 v2:172.21.15.171:6838/1082475840 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556966a857b0 con 0x556966b2ae90 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f128effd640 10 client.25142 handle_command_reply: tid=0 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 client.25142 shutdown 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 -- 172.21.15.49:0/3815868584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 msgr2=0x556966b1e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b1e690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f12a4098320 tx=0x7f12a409a090 comp rx=0 tx=0).stop 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 -- 172.21.15.49:0/3815868584 shutdown_connections 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x556966b2ae90 0x556966b2c300 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 0x556966b256d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b14e10 0x556966b1e690 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 --2- 172.21.15.49:0/3815868584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 0x556966b1e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f128effd640 0 client.25142 ms_handle_reset on v2:172.21.15.171:6838/1082475840 2024-03-26T18:01:31.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 -- 172.21.15.49:0/3815868584 >> 172.21.15.49:0/3815868584 conn(0x556966b0f030 msgr2=0x556966b0e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 -- 172.21.15.49:0/3815868584 shutdown_connections 2024-03-26T18:01:31.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.396+0000 7f12b89ab740 1 -- 172.21.15.49:0/3815868584 wait complete. 2024-03-26T18:01:31.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.397+0000 7f12b89ab740 20 client.25142 trim_cache size 0 max 16384 2024-03-26T18:01:31.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12b89ab740 1 Processor -- start 2024-03-26T18:01:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12b89ab740 1 -- start start 2024-03-26T18:01:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b13c40 0x556966b188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b14e10 0x556966b18e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 0x556966b1e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12b89ab740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556966b2a640 con 0x556966b157d0 2024-03-26T18:01:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12b89ab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556966b2ad10 con 0x556966b13c40 2024-03-26T18:01:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12b89ab740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556966b2b3e0 con 0x556966b14e10 2024-03-26T18:01:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 0x556966b1e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12affff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 0x556966b1e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60216/0 (socket says 172.21.15.49:60216) 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12affff640 1 -- 172.21.15.49:0/3292404642 learned_addr learned my addr 172.21.15.49:0/3292404642 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12af7fe640 1 --2- 172.21.15.49:0/3292404642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b13c40 0x556966b188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12aeffd640 1 --2- 172.21.15.49:0/3292404642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b14e10 0x556966b18e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12affff640 1 -- 172.21.15.49:0/3292404642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b14e10 msgr2=0x556966b18e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12affff640 1 --2- 172.21.15.49:0/3292404642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b14e10 0x556966b18e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12affff640 1 -- 172.21.15.49:0/3292404642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b13c40 msgr2=0x556966b188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12affff640 1 --2- 172.21.15.49:0/3292404642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b13c40 0x556966b188e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12affff640 1 -- 172.21.15.49:0/3292404642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556966b14890 con 0x556966b157d0 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12af7fe640 1 --2- 172.21.15.49:0/3292404642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b13c40 0x556966b188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.415+0000 7f12affff640 1 --2- 172.21.15.49:0/3292404642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 0x556966b1e150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f12a802c700 tx=0x7f12a802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2d2a3a77ac8f20 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12957fa640 1 -- 172.21.15.49:0/3292404642 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8039040 con 0x556966b157d0 2024-03-26T18:01:31.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12957fa640 1 -- 172.21.15.49:0/3292404642 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a8053070 con 0x556966b157d0 2024-03-26T18:01:31.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12957fa640 1 -- 172.21.15.49:0/3292404642 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a804e6a0 con 0x556966b157d0 2024-03-26T18:01:31.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12b89ab740 1 -- 172.21.15.49:0/3292404642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 msgr2=0x556966b1e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12b89ab740 1 --2- 172.21.15.49:0/3292404642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 0x556966b1e150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f12a802c700 tx=0x7f12a802e040 comp rx=0 tx=0).stop 2024-03-26T18:01:31.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12b89ab740 1 -- 172.21.15.49:0/3292404642 shutdown_connections 2024-03-26T18:01:31.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12b89ab740 1 --2- 172.21.15.49:0/3292404642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b14e10 0x556966b18e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12b89ab740 1 --2- 172.21.15.49:0/3292404642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b157d0 0x556966b1e150 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12b89ab740 1 --2- 172.21.15.49:0/3292404642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b13c40 0x556966b188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12b89ab740 1 -- 172.21.15.49:0/3292404642 >> 172.21.15.49:0/3292404642 conn(0x556966b0f030 msgr2=0x556966b1d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12b89ab740 1 -- 172.21.15.49:0/3292404642 shutdown_connections 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12b89ab740 1 -- 172.21.15.49:0/3292404642 wait complete. 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.416+0000 7f12b89ab740 1 -- start start 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12b89ab740 1 Processor -- start 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 0x556966b8ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b14e10 0x556966b8d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12b89ab740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b157d0 0x556966b93cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12b89ab740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556966b2c180 con 0x556966b14e10 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12b89ab740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x556966b2bab0 con 0x556966b157d0 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12b89ab740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556966b2c850 con 0x556966b13c40 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12b89ab740 10 client.0 _ll_register_callbacks cb 0x7f12a409bd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b14e10 0x556966b8d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b14e10 0x556966b8d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60228/0 (socket says 172.21.15.49:60228) 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12af7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 0x556966b8ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12aeffd640 1 -- 172.21.15.49:0/6817162 learned_addr learned my addr 172.21.15.49:0/6817162 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12affff640 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b157d0 0x556966b93cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12aeffd640 1 -- 172.21.15.49:0/6817162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 msgr2=0x556966b8ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12aeffd640 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 0x556966b8ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12aeffd640 1 -- 172.21.15.49:0/6817162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b157d0 msgr2=0x556966b93cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12aeffd640 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b157d0 0x556966b93cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12aeffd640 1 -- 172.21.15.49:0/6817162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556966b94270 con 0x556966b14e10 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.417+0000 7f12af7fe640 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 0x556966b8ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f12aeffd640 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b14e10 0x556966b8d370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f12a400c6a0 tx=0x7f12a409a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982f15dd95cf122c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f128effd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f128effd640 1 -- 172.21.15.49:0/6817162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4098160 con 0x556966b14e10 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f128effd640 1 -- 172.21.15.49:0/6817162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f12a40b6050 con 0x556966b14e10 2024-03-26T18:01:31.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f12b89ab740 1 -- 172.21.15.49:0/6817162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a8053070 con 0x556966b14e10 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f128effd640 1 -- 172.21.15.49:0/6817162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f12a409ba90 con 0x556966b14e10 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f12b89ab740 10 client.15252 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f12b89ab740 1 -- 172.21.15.49:0/6817162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556966b95570 con 0x556966b14e10 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f128effd640 1 -- 172.21.15.49:0/6817162 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a8053070 con 0x556966b14e10 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f128effd640 1 -- 172.21.15.49:0/6817162 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f12a40b8070 con 0x556966b14e10 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f12b89ab740 10 client.15252 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f12b89ab740 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f12b89ab740 20 client.15252 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f12b89ab740 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x556966b9a0e0 0x556966b96bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f12b89ab740 4 client.15252 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.418+0000 7f12b89ab740 1 -- 172.21.15.49:0/6817162 --> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] -- command(tid 0: {"prefix": "status"}) -- 0x556966a7fb00 con 0x556966b9a0e0 2024-03-26T18:01:31.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12af7fe640 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x556966b9a0e0 0x556966b96bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f128effd640 10 client.15252 ms_handle_connect on v2:172.21.15.171:6838/1082475840 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12af7fe640 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x556966b9a0e0 0x556966b96bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c5cd45f3383a772 server_cookie=939e73d241149550 in_seq=0 out_seq=0 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f128effd640 1 -- 172.21.15.49:0/6817162 <== mds.2 v2:172.21.15.171:6838/1082475840 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556966a7fb00 con 0x556966b9a0e0 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f128effd640 10 client.15252 handle_command_reply: tid=0 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12b89ab740 1 client.15252 shutdown 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12b89ab740 1 -- 172.21.15.49:0/6817162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b14e10 msgr2=0x556966b8d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12b89ab740 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b14e10 0x556966b8d370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f12a400c6a0 tx=0x7f12a409a1c0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12b89ab740 1 -- 172.21.15.49:0/6817162 shutdown_connections 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12b89ab740 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x556966b9a0e0 0x556966b96bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12b89ab740 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x556966b157d0 0x556966b93cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f128effd640 0 client.15252 ms_handle_reset on v2:172.21.15.171:6838/1082475840 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12b89ab740 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556966b14e10 0x556966b8d370 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12b89ab740 1 --2- 172.21.15.49:0/6817162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556966b13c40 0x556966b8ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.419+0000 7f12b89ab740 1 -- 172.21.15.49:0/6817162 >> 172.21.15.49:0/6817162 conn(0x556966b0f030 msgr2=0x556966b188e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b89ab740 1 -- 172.21.15.49:0/6817162 shutdown_connections 2024-03-26T18:01:31.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b89ab740 1 -- 172.21.15.49:0/6817162 wait complete. 2024-03-26T18:01:31.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b89ab740 20 client.15252 trim_cache size 0 max 16384 2024-03-26T18:01:31.410 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:01:31.410 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T18:01:31.410 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2024-03-26T18:01:31.411 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24478, 2024-03-26T18:01:31.411 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T18:01:31.411 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T18:01:31.411 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T18:01:31.411 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 84.908938101000004, 2024-03-26T18:01:31.411 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2024-03-26T18:01:31.411 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 92, 2024-03-26T18:01:31.411 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T18:01:31.411 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 116.797974465 2024-03-26T18:01:31.411 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:01:31.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 -- 172.21.15.49:0/299268456 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12980a2f20 msgr2=0x7f12980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12980a2f20 0x7f12980a53e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x556966b14d10 tx=0x7f12a802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 -- 172.21.15.49:0/299268456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b0102ad0 msgr2=0x7f12b0198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b0102ad0 0x7f12b0198160 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f12a400e9f0 tx=0x7f12a400eeb0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 -- 172.21.15.49:0/299268456 shutdown_connections 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12980a2f20 0x7f12980a53e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b0103490 0x7f12b019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b0102ad0 0x7f12b0198160 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 --2- 172.21.15.49:0/299268456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b01018e0 0x7f12b0197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 -- 172.21.15.49:0/299268456 >> 172.21.15.49:0/299268456 conn(0x7f12b00fce50 msgr2=0x7f12b0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 -- 172.21.15.49:0/299268456 shutdown_connections 2024-03-26T18:01:31.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.420+0000 7f12b5dc4640 1 -- 172.21.15.49:0/299268456 wait complete. 2024-03-26T18:01:31.414 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-26T18:01:31.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.542+0000 7feb11d52640 1 -- 172.21.15.49:0/4058029075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb0c1075c0 msgr2=0x7feb0c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.542+0000 7feb11d52640 1 --2- 172.21.15.49:0/4058029075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb0c1075c0 0x7feb0c109a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feb04009be0 tx=0x7feb0402bef0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.542+0000 7feb11d52640 1 -- 172.21.15.49:0/4058029075 shutdown_connections 2024-03-26T18:01:31.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.542+0000 7feb11d52640 1 --2- 172.21.15.49:0/4058029075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb0c1075c0 0x7feb0c109a10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.542+0000 7feb11d52640 1 --2- 172.21.15.49:0/4058029075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb0c104c90 0x7feb0c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.542+0000 7feb11d52640 1 --2- 172.21.15.49:0/4058029075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb0c102360 0x7feb0c104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.542+0000 7feb11d52640 1 -- 172.21.15.49:0/4058029075 >> 172.21.15.49:0/4058029075 conn(0x7feb0c0fbd70 msgr2=0x7feb0c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.542+0000 7feb11d52640 1 -- 172.21.15.49:0/4058029075 shutdown_connections 2024-03-26T18:01:31.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.542+0000 7feb11d52640 1 -- 172.21.15.49:0/4058029075 wait complete. 2024-03-26T18:01:31.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb11d52640 1 Processor -- start 2024-03-26T18:01:31.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb11d52640 1 -- start start 2024-03-26T18:01:31.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb11d52640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb0c102360 0x7feb0c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb11d52640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb0c104c90 0x7feb0c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb11d52640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb0c1075c0 0x7feb0c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb11d52640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c10d580 con 0x7feb0c1075c0 2024-03-26T18:01:31.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb11d52640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c10d400 con 0x7feb0c104c90 2024-03-26T18:01:31.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb11d52640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c10d700 con 0x7feb0c102360 2024-03-26T18:01:31.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb0c1075c0 0x7feb0c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb0c1075c0 0x7feb0c19f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60248/0 (socket says 172.21.15.49:60248) 2024-03-26T18:01:31.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb0bfff640 1 -- 172.21.15.49:0/3012813056 learned_addr learned my addr 172.21.15.49:0/3012813056 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb0affd640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb0c104c90 0x7feb0c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb0c102360 0x7feb0c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb0bfff640 1 -- 172.21.15.49:0/3012813056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb0c102360 msgr2=0x7feb0c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb0bfff640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb0c102360 0x7feb0c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb0bfff640 1 -- 172.21.15.49:0/3012813056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb0c104c90 msgr2=0x7feb0c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb0bfff640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb0c104c90 0x7feb0c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.543+0000 7feb0bfff640 1 -- 172.21.15.49:0/3012813056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c19f7e0 con 0x7feb0c1075c0 2024-03-26T18:01:31.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.544+0000 7feb0bfff640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb0c1075c0 0x7feb0c19f170 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feb04036040 tx=0x7feb040337b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.544+0000 7feb08ff9640 1 -- 172.21.15.49:0/3012813056 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04043020 con 0x7feb0c1075c0 2024-03-26T18:01:31.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.544+0000 7feb11d52640 1 -- 172.21.15.49:0/3012813056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c19fa70 con 0x7feb0c1075c0 2024-03-26T18:01:31.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.544+0000 7feb11d52640 1 -- 172.21.15.49:0/3012813056 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c1a0010 con 0x7feb0c1075c0 2024-03-26T18:01:31.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.544+0000 7feb08ff9640 1 -- 172.21.15.49:0/3012813056 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb04003b80 con 0x7feb0c1075c0 2024-03-26T18:01:31.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.544+0000 7feb08ff9640 1 -- 172.21.15.49:0/3012813056 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04034690 con 0x7feb0c1075c0 2024-03-26T18:01:31.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.546+0000 7feb08ff9640 1 -- 172.21.15.49:0/3012813056 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7feb04046020 con 0x7feb0c1075c0 2024-03-26T18:01:31.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.546+0000 7feb14a0e740 1 Processor -- start 2024-03-26T18:01:31.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.546+0000 7feb14a0e740 1 -- start start 2024-03-26T18:01:31.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d0e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d0f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 0x5592c6d16470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb14a0e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5592c6c6e270 con 0x5592c6d0fbe0 2024-03-26T18:01:31.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb14a0e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5592c6c16c80 con 0x5592c6d0e050 2024-03-26T18:01:31.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb14a0e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5592c6c6a220 con 0x5592c6d0f220 2024-03-26T18:01:31.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb08ff9640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7feaf80a2f20 0x7feaf80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d0e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d0f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d0e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:38462/0 (socket says 172.21.15.49:38462) 2024-03-26T18:01:31.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0affd640 1 -- 172.21.15.49:0/175907987 learned_addr learned my addr 172.21.15.49:0/175907987 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb08ff9640 1 -- 172.21.15.49:0/3012813056 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7feb040f1730 con 0x7feb0c1075c0 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0affd640 1 --2- 172.21.15.49:0/175907987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 0x5592c6d16470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0bfff640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7feaf80a2f20 0x7feaf80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0b7fe640 1 -- 172.21.15.49:0/175907987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 msgr2=0x5592c6d0e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/175907987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d0e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0b7fe640 1 -- 172.21.15.49:0/175907987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 msgr2=0x5592c6d16470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/175907987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 0x5592c6d16470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0b7fe640 1 -- 172.21.15.49:0/175907987 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c6c28fb0 con 0x5592c6d0f220 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0bfff640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7feaf80a2f20 0x7feaf80a53e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feb04033ff0 tx=0x7feb04009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.547+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/175907987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d0f6a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009c00 tx=0x7feaf4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2547d042867e2db server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb01ffb640 1 -- 172.21.15.49:0/175907987 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4004050 con 0x5592c6d0f220 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb01ffb640 1 -- 172.21.15.49:0/175907987 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf4005500 con 0x5592c6d0f220 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb01ffb640 1 -- 172.21.15.49:0/175907987 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4004e10 con 0x5592c6d0f220 2024-03-26T18:01:31.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb14a0e740 1 -- 172.21.15.49:0/175907987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 msgr2=0x5592c6d0f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb14a0e740 1 --2- 172.21.15.49:0/175907987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d0f6a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009c00 tx=0x7feaf4011810 comp rx=0 tx=0).stop 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb14a0e740 1 -- 172.21.15.49:0/175907987 shutdown_connections 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb14a0e740 1 --2- 172.21.15.49:0/175907987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d0f6a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb14a0e740 1 --2- 172.21.15.49:0/175907987 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 0x5592c6d16470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb14a0e740 1 --2- 172.21.15.49:0/175907987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d0e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb14a0e740 1 -- 172.21.15.49:0/175907987 >> 172.21.15.49:0/175907987 conn(0x5592c6d05030 msgr2=0x5592c6d05430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb14a0e740 1 -- 172.21.15.49:0/175907987 shutdown_connections 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.548+0000 7feb14a0e740 1 -- 172.21.15.49:0/175907987 wait complete. 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb14a0e740 1 -- start start 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb14a0e740 1 Processor -- start 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d18570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d18ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 0x5592c6d1faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb14a0e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5592c6d16bf0 con 0x5592c6d0fbe0 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb14a0e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5592c6c29fc0 con 0x5592c6d0e050 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb14a0e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5592c6d16d70 con 0x5592c6d0f220 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb14a0e740 10 client.0 _ll_register_callbacks cb 0x5592c6c24770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d18570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d18ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb0affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 0x5592c6d1faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d18ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:42882/0 (socket says 172.21.15.49:42882) 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.549+0000 7feb0b7fe640 1 -- 172.21.15.49:0/1589048877 learned_addr learned my addr 172.21.15.49:0/1589048877 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feb0affd640 1 -- 172.21.15.49:0/1589048877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 msgr2=0x5592c6d18ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feb0affd640 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d18ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feb0affd640 1 -- 172.21.15.49:0/1589048877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 msgr2=0x5592c6d18570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feb0affd640 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d18570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feb0affd640 1 -- 172.21.15.49:0/1589048877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c69822d0 con 0x5592c6d0fbe0 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d18ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feb0affd640 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 0x5592c6d1faf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feafc009fd0 tx=0x7feafc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8471fc671d1dfe2c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feaeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feaeaffd640 1 -- 172.21.15.49:0/1589048877 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc00fe30 con 0x5592c6d0fbe0 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feaeaffd640 1 -- 172.21.15.49:0/1589048877 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7feafc005020 con 0x5592c6d0fbe0 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feb14a0e740 1 -- 172.21.15.49:0/1589048877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf4005500 con 0x5592c6d0fbe0 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feaeaffd640 1 -- 172.21.15.49:0/1589048877 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feafc004020 con 0x5592c6d0fbe0 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feb14a0e740 10 client.15264 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.550+0000 7feb14a0e740 1 -- 172.21.15.49:0/1589048877 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592c6a4c6b0 con 0x5592c6d0fbe0 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feaeaffd640 1 -- 172.21.15.49:0/1589048877 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf4005500 con 0x5592c6d0fbe0 2024-03-26T18:01:31.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feaeaffd640 1 -- 172.21.15.49:0/1589048877 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feafc0047a0 con 0x5592c6d0fbe0 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feb14a0e740 10 client.15264 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feb14a0e740 10 client.15264 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feb14a0e740 20 client.15264 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5592c6d25420 0x5592c6d26890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feb14a0e740 4 client.15264 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feb14a0e740 1 -- 172.21.15.49:0/1589048877 --> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c6c7b7b0 con 0x5592c6d25420 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5592c6d25420 0x5592c6d26890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feaeaffd640 10 client.15264 ms_handle_connect on v2:172.21.15.115:6836/1993258438 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.551+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5592c6d25420 0x5592c6d26890 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e38495a69ced3e6 server_cookie=a9addcbb84387bf0 in_seq=0 out_seq=0 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.552+0000 7feaeaffd640 1 -- 172.21.15.49:0/1589048877 <== mds.1 v2:172.21.15.115:6836/1993258438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5592c6c7b7b0 con 0x5592c6d25420 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.552+0000 7feaeaffd640 10 client.15264 handle_command_reply: tid=0 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.552+0000 7feb14a0e740 1 client.15264 shutdown 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.552+0000 7feb14a0e740 1 -- 172.21.15.49:0/1589048877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 msgr2=0x5592c6d1faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.552+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 0x5592c6d1faf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feafc009fd0 tx=0x7feafc009260 comp rx=0 tx=0).stop 2024-03-26T18:01:31.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.552+0000 7feb14a0e740 1 -- 172.21.15.49:0/1589048877 shutdown_connections 2024-03-26T18:01:31.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.552+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5592c6d25420 0x5592c6d26890 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.552+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d18ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.552+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0fbe0 0x5592c6d1faf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.552+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1589048877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d18570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.553+0000 7feb14a0e740 1 -- 172.21.15.49:0/1589048877 >> 172.21.15.49:0/1589048877 conn(0x5592c6d05030 msgr2=0x5592c6d04230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.553+0000 7feaeaffd640 0 client.15264 ms_handle_reset on v2:172.21.15.115:6836/1993258438 2024-03-26T18:01:31.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.553+0000 7feb14a0e740 1 -- 172.21.15.49:0/1589048877 shutdown_connections 2024-03-26T18:01:31.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.553+0000 7feb14a0e740 1 -- 172.21.15.49:0/1589048877 wait complete. 2024-03-26T18:01:31.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.553+0000 7feb14a0e740 20 client.15264 trim_cache size 0 max 16384 2024-03-26T18:01:31.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.571+0000 7feb14a0e740 1 Processor -- start 2024-03-26T18:01:31.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.571+0000 7feb14a0e740 1 -- start start 2024-03-26T18:01:31.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.571+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0e050 0x5592c6d16ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.571+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d1fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.571+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0fbe0 0x5592c6cffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.571+0000 7feb14a0e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5592c6d24bd0 con 0x5592c6d0e050 2024-03-26T18:01:31.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.571+0000 7feb14a0e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5592c6d252a0 con 0x5592c6d0fbe0 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.571+0000 7feb14a0e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5592c6d25970 con 0x5592c6d0f220 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0e050 0x5592c6d16ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d1fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d1fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:42886/0 (socket says 172.21.15.49:42886) 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0e050 0x5592c6d16ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60282/0 (socket says 172.21.15.49:60282) 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0b7fe640 1 -- 172.21.15.49:0/3980271960 learned_addr learned my addr 172.21.15.49:0/3980271960 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0affd640 1 --2- 172.21.15.49:0/3980271960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0fbe0 0x5592c6cffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0b7fe640 1 -- 172.21.15.49:0/3980271960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0fbe0 msgr2=0x5592c6cffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/3980271960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0fbe0 0x5592c6cffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0b7fe640 1 -- 172.21.15.49:0/3980271960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0e050 msgr2=0x5592c6d16ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/3980271960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0e050 0x5592c6d16ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0b7fe640 1 -- 172.21.15.49:0/3980271960 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c6d0eca0 con 0x5592c6d0f220 2024-03-26T18:01:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/3980271960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d1fab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feaf4012fd0 tx=0x7feaf40096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9afe00dac81c04d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb017fa640 1 -- 172.21.15.49:0/3980271960 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4004360 con 0x5592c6d0f220 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb017fa640 1 -- 172.21.15.49:0/3980271960 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf4004500 con 0x5592c6d0f220 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.572+0000 7feb017fa640 1 -- 172.21.15.49:0/3980271960 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4003850 con 0x5592c6d0f220 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 -- 172.21.15.49:0/3980271960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 msgr2=0x5592c6d1fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 --2- 172.21.15.49:0/3980271960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d1fab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feaf4012fd0 tx=0x7feaf40096a0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 -- 172.21.15.49:0/3980271960 shutdown_connections 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 --2- 172.21.15.49:0/3980271960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0f220 0x5592c6d1fab0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 --2- 172.21.15.49:0/3980271960 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0fbe0 0x5592c6cffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 --2- 172.21.15.49:0/3980271960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0e050 0x5592c6d16ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 -- 172.21.15.49:0/3980271960 >> 172.21.15.49:0/3980271960 conn(0x5592c6d05030 msgr2=0x5592c6d05410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 -- 172.21.15.49:0/3980271960 shutdown_connections 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 -- 172.21.15.49:0/3980271960 wait complete. 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 -- start start 2024-03-26T18:01:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.573+0000 7feb14a0e740 1 Processor -- start 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d14c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0f220 0x5592c6d15170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb14a0e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0fbe0 0x5592c6d89180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb14a0e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5592c6d26710 con 0x5592c6d0f220 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb14a0e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5592c6d26040 con 0x5592c6d0e050 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb14a0e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5592c6d26de0 con 0x5592c6d0fbe0 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb14a0e740 10 client.0 _ll_register_callbacks cb 0x7feafc011b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0f220 0x5592c6d15170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0f220 0x5592c6d15170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60290/0 (socket says 172.21.15.49:60290) 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0b7fe640 1 -- 172.21.15.49:0/1010217438 learned_addr learned my addr 172.21.15.49:0/1010217438 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0affd640 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d14c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0affd640 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0fbe0 0x5592c6d89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0b7fe640 1 -- 172.21.15.49:0/1010217438 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0fbe0 msgr2=0x5592c6d89180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0fbe0 0x5592c6d89180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0b7fe640 1 -- 172.21.15.49:0/1010217438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 msgr2=0x5592c6d14c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d14c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0b7fe640 1 -- 172.21.15.49:0/1010217438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c6d896c0 con 0x5592c6d0f220 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feb0b7fe640 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0f220 0x5592c6d15170 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009250 tx=0x7feaf40097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af58823f7a3ee5e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feaeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.574+0000 7feaeaffd640 1 -- 172.21.15.49:0/1010217438 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4004360 con 0x5592c6d0f220 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feaeaffd640 1 -- 172.21.15.49:0/1010217438 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7feaf402eab0 con 0x5592c6d0f220 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feb14a0e740 1 -- 172.21.15.49:0/1010217438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf4004500 con 0x5592c6d0f220 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feaeaffd640 1 -- 172.21.15.49:0/1010217438 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feaf402ecb0 con 0x5592c6d0f220 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feb14a0e740 10 client.15270 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feb14a0e740 1 -- 172.21.15.49:0/1010217438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592c6d8a8c0 con 0x5592c6d0f220 2024-03-26T18:01:31.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feaeaffd640 1 -- 172.21.15.49:0/1010217438 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf4004500 con 0x5592c6d0f220 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feaeaffd640 1 -- 172.21.15.49:0/1010217438 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feaf4037460 con 0x5592c6d0f220 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feb14a0e740 10 client.15270 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feb14a0e740 10 client.15270 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feb14a0e740 20 client.15270 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5592c6d8eeb0 0x5592c6d8dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feb14a0e740 4 client.15270 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feb14a0e740 1 -- 172.21.15.49:0/1010217438 --> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] -- command(tid 0: {"prefix": "status"}) -- 0x5592c6c75b00 con 0x5592c6d8eeb0 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.575+0000 7feb0affd640 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5592c6d8eeb0 0x5592c6d8dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feaeaffd640 10 client.15270 ms_handle_connect on v2:172.21.15.115:6836/1993258438 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb0affd640 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5592c6d8eeb0 0x5592c6d8dbc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db1225131c3b2357 server_cookie=2c24c2d5925442cb in_seq=0 out_seq=0 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feaeaffd640 1 -- 172.21.15.49:0/1010217438 <== mds.1 v2:172.21.15.115:6836/1993258438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5592c6c75b00 con 0x5592c6d8eeb0 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feaeaffd640 10 client.15270 handle_command_reply: tid=0 2024-03-26T18:01:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb14a0e740 1 client.15270 shutdown 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb14a0e740 1 -- 172.21.15.49:0/1010217438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0f220 msgr2=0x5592c6d15170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0f220 0x5592c6d15170 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009250 tx=0x7feaf40097d0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb14a0e740 1 -- 172.21.15.49:0/1010217438 shutdown_connections 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5592c6d8eeb0 0x5592c6d8dbc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592c6d0fbe0 0x5592c6d89180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5592c6d0f220 0x5592c6d15170 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feaeaffd640 0 client.15270 ms_handle_reset on v2:172.21.15.115:6836/1993258438 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb14a0e740 1 --2- 172.21.15.49:0/1010217438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5592c6d0e050 0x5592c6d14c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb14a0e740 1 -- 172.21.15.49:0/1010217438 >> 172.21.15.49:0/1010217438 conn(0x5592c6d05030 msgr2=0x5592c6d05410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.576+0000 7feb14a0e740 1 -- 172.21.15.49:0/1010217438 shutdown_connections 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb14a0e740 1 -- 172.21.15.49:0/1010217438 wait complete. 2024-03-26T18:01:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb14a0e740 20 client.15270 trim_cache size 0 max 16384 2024-03-26T18:01:31.566 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:01:31.566 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T18:01:31.566 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2024-03-26T18:01:31.566 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24503, 2024-03-26T18:01:31.566 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T18:01:31.566 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T18:01:31.566 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T18:01:31.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 86.072638402999999, 2024-03-26T18:01:31.566 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2024-03-26T18:01:31.566 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 92, 2024-03-26T18:01:31.567 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T18:01:31.567 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 119.729826398 2024-03-26T18:01:31.567 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:01:31.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 -- 172.21.15.49:0/3012813056 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7feaf80a2f20 msgr2=0x7feaf80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7feaf80a2f20 0x7feaf80a53e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feb04033ff0 tx=0x7feb04009500 comp rx=0 tx=0).stop 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 -- 172.21.15.49:0/3012813056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb0c1075c0 msgr2=0x7feb0c19f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb0c1075c0 0x7feb0c19f170 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feb04036040 tx=0x7feb040337b0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 -- 172.21.15.49:0/3012813056 shutdown_connections 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7feaf80a2f20 0x7feaf80a53e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb0c1075c0 0x7feb0c19f170 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb0c104c90 0x7feb0c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 --2- 172.21.15.49:0/3012813056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb0c102360 0x7feb0c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 -- 172.21.15.49:0/3012813056 >> 172.21.15.49:0/3012813056 conn(0x7feb0c0fbd70 msgr2=0x7feb0c102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 -- 172.21.15.49:0/3012813056 shutdown_connections 2024-03-26T18:01:31.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.577+0000 7feb11d52640 1 -- 172.21.15.49:0/3012813056 wait complete. 2024-03-26T18:01:31.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:31 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2876729543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:31.569 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:31 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1519262994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:31.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T18:01:31.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:31 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2876729543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:31.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:31 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1519262994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.698+0000 7f5fb283d640 1 -- 172.21.15.49:0/2376900405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fac103480 msgr2=0x7f5fac109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.698+0000 7f5fb283d640 1 --2- 172.21.15.49:0/2376900405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fac103480 0x7f5fac109d10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009ca0 tx=0x7f5fa402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.698+0000 7f5fb283d640 1 -- 172.21.15.49:0/2376900405 shutdown_connections 2024-03-26T18:01:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.698+0000 7f5fb283d640 1 --2- 172.21.15.49:0/2376900405 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fac103480 0x7f5fac109d10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.698+0000 7f5fb283d640 1 --2- 172.21.15.49:0/2376900405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fac102ac0 0x7f5fac102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.698+0000 7f5fb283d640 1 --2- 172.21.15.49:0/2376900405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fac1018d0 0x7f5fac101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.698+0000 7f5fb283d640 1 -- 172.21.15.49:0/2376900405 >> 172.21.15.49:0/2376900405 conn(0x7f5fac0fce60 msgr2=0x7f5fac0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.698+0000 7f5fb283d640 1 -- 172.21.15.49:0/2376900405 shutdown_connections 2024-03-26T18:01:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.698+0000 7f5fb283d640 1 -- 172.21.15.49:0/2376900405 wait complete. 2024-03-26T18:01:31.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fb283d640 1 Processor -- start 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fb283d640 1 -- start start 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fb283d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fac1018d0 0x7f5fac197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fb283d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fac102ac0 0x7f5fac1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fb283d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fac103480 0x7f5fac19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fb283d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac10d530 con 0x7f5fac1018d0 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fb283d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac10d3b0 con 0x7f5fac103480 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fb283d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac10d6b0 con 0x7f5fac102ac0 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fac1018d0 0x7f5fac197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fac1018d0 0x7f5fac197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60306/0 (socket says 172.21.15.49:60306) 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fabfff640 1 -- 172.21.15.49:0/1695388402 learned_addr learned my addr 172.21.15.49:0/1695388402 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fb0dda640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fac103480 0x7f5fac19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.699+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fac102ac0 0x7f5fac1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fb0dda640 1 -- 172.21.15.49:0/1695388402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fac102ac0 msgr2=0x7f5fac1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fb0dda640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fac102ac0 0x7f5fac1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fb0dda640 1 -- 172.21.15.49:0/1695388402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fac1018d0 msgr2=0x7f5fac197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fb0dda640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fac1018d0 0x7f5fac197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fb0dda640 1 -- 172.21.15.49:0/1695388402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac19f790 con 0x7f5fac103480 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fabfff640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fac1018d0 0x7f5fac197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fb0dda640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fac103480 0x7f5fac19f120 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402c630 tx=0x7f5fa4032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fa97fa640 1 -- 172.21.15.49:0/1695388402 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4041020 con 0x7f5fac103480 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fa97fa640 1 -- 172.21.15.49:0/1695388402 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa4034cd0 con 0x7f5fac103480 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fa97fa640 1 -- 172.21.15.49:0/1695388402 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4039040 con 0x7f5fac103480 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fb283d640 1 -- 172.21.15.49:0/1695388402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac19fa20 con 0x7f5fac103480 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.700+0000 7f5fb283d640 1 -- 172.21.15.49:0/1695388402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac19fee0 con 0x7f5fac103480 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.702+0000 7f5fa97fa640 1 -- 172.21.15.49:0/1695388402 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f5fa4044020 con 0x7f5fac103480 2024-03-26T18:01:31.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.702+0000 7f5fb5421740 1 Processor -- start 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.702+0000 7f5fb5421740 1 -- start start 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0ca4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0ca5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca57d0 0x5624a0cac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fa97fa640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5f940a2f20 0x7f5f940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fb5421740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5624a0c08270 con 0x5624a0ca4e10 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fb5421740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5624a0bb0c80 con 0x5624a0ca3c40 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fb5421740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5624a0c04220 con 0x5624a0ca57d0 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0ca5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0ca5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60316/0 (socket says 172.21.15.49:60316) 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fabfff640 1 -- 172.21.15.49:0/713531170 learned_addr learned my addr 172.21.15.49:0/713531170 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/713531170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0ca4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fb0dda640 1 --2- 172.21.15.49:0/713531170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca57d0 0x5624a0cac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5f940a2f20 0x7f5f940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fa97fa640 1 -- 172.21.15.49:0/1695388402 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f5fa40f14b0 con 0x7f5fac103480 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fabfff640 1 -- 172.21.15.49:0/713531170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca57d0 msgr2=0x5624a0cac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fabfff640 1 --2- 172.21.15.49:0/713531170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca57d0 0x5624a0cac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fabfff640 1 -- 172.21.15.49:0/713531170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 msgr2=0x5624a0ca4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fabfff640 1 --2- 172.21.15.49:0/713531170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0ca4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fabfff640 1 -- 172.21.15.49:0/713531170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624a0bc2fb0 con 0x5624a0ca4e10 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/713531170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0ca4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fabfff640 1 --2- 172.21.15.49:0/713531170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0ca5290 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0115f0 tx=0x7f5f9c0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed0c2c65c059acb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5f940a2f20 0x7f5f940a53e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x5624a0ca4d10 tx=0x7f5fa0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.703+0000 7f5f927fc640 1 -- 172.21.15.49:0/713531170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c01e070 con 0x5624a0ca4e10 2024-03-26T18:01:31.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5f927fc640 1 -- 172.21.15.49:0/713531170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f9c00fac0 con 0x5624a0ca4e10 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5f927fc640 1 -- 172.21.15.49:0/713531170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c019c30 con 0x5624a0ca4e10 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5fb5421740 1 -- 172.21.15.49:0/713531170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 msgr2=0x5624a0ca5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5fb5421740 1 --2- 172.21.15.49:0/713531170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0ca5290 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0115f0 tx=0x7f5f9c0119f0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5fb5421740 1 -- 172.21.15.49:0/713531170 shutdown_connections 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5fb5421740 1 --2- 172.21.15.49:0/713531170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca57d0 0x5624a0cac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5fb5421740 1 --2- 172.21.15.49:0/713531170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0ca5290 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5fb5421740 1 --2- 172.21.15.49:0/713531170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0ca4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5fb5421740 1 -- 172.21.15.49:0/713531170 >> 172.21.15.49:0/713531170 conn(0x5624a0c9f030 msgr2=0x5624a0c9f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5fb5421740 1 -- 172.21.15.49:0/713531170 shutdown_connections 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.704+0000 7f5fb5421740 1 -- 172.21.15.49:0/713531170 wait complete. 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fb5421740 1 -- start start 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fb5421740 1 Processor -- start 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 0x5624a0cae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca57d0 0x5624a0cb5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fb5421740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5624a0cac7e0 con 0x5624a0ca3c40 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fb5421740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5624a0bc3fc0 con 0x5624a0ca57d0 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fb5421740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5624a0cac960 con 0x5624a0ca4e10 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fb5421740 10 client.0 _ll_register_callbacks cb 0x5624a0bbe770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60330/0 (socket says 172.21.15.49:60330) 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fabfff640 1 -- 172.21.15.49:0/2202404508 learned_addr learned my addr 172.21.15.49:0/2202404508 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 0x5624a0cae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fb0dda640 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca57d0 0x5624a0cb5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fabfff640 1 -- 172.21.15.49:0/2202404508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 msgr2=0x5624a0cae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fabfff640 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 0x5624a0cae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fabfff640 1 -- 172.21.15.49:0/2202404508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca57d0 msgr2=0x5624a0cb5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.705+0000 7f5fabfff640 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca57d0 0x5624a0cb5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5fabfff640 1 -- 172.21.15.49:0/2202404508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624a091c2d0 con 0x5624a0ca3c40 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5fabfff640 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cae1e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009520 tx=0x7f5f9c00f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf0f0ce32e31dcf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5f8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/2202404508 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c01e040 con 0x5624a0ca3c40 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/2202404508 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f5f9c019970 con 0x5624a0ca3c40 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5fb5421740 1 -- 172.21.15.49:0/2202404508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f9c00fac0 con 0x5624a0ca3c40 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/2202404508 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f9c005c50 con 0x5624a0ca3c40 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5fb5421740 10 client.15282 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5fb5421740 1 -- 172.21.15.49:0/2202404508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624a09e66b0 con 0x5624a0ca3c40 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/2202404508 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f9c00fac0 con 0x5624a0ca3c40 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/2202404508 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5f9c021070 con 0x5624a0ca3c40 2024-03-26T18:01:31.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5fb5421740 10 client.15282 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.706+0000 7f5fb5421740 10 client.15282 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.707+0000 7f5fb5421740 20 client.15282 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.707+0000 7f5fb5421740 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5624a0cbade0 0x5624a0cbc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.707+0000 7f5fb5421740 4 client.15282 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.707+0000 7f5fb5421740 1 -- 172.21.15.49:0/2202404508 --> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624a0c157b0 con 0x5624a0cbade0 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.707+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5624a0cbade0 0x5624a0cbc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.707+0000 7f5f8b7fe640 10 client.15282 ms_handle_connect on v2:172.21.15.115:6838/1435040687 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.707+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5624a0cbade0 0x5624a0cbc250 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7be38c2b437bab server_cookie=9a36d226a1651e01 in_seq=0 out_seq=0 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/2202404508 <== mds.0 v2:172.21.15.115:6838/1435040687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5624a0c157b0 con 0x5624a0cbade0 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5f8b7fe640 10 client.15282 handle_command_reply: tid=0 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 client.15282 shutdown 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 -- 172.21.15.49:0/2202404508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 msgr2=0x5624a0cae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cae1e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009520 tx=0x7f5f9c00f890 comp rx=0 tx=0).stop 2024-03-26T18:01:31.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 -- 172.21.15.49:0/2202404508 shutdown_connections 2024-03-26T18:01:31.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5624a0cbade0 0x5624a0cbc250 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 0x5624a0cae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca57d0 0x5624a0cb5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5f8b7fe640 0 client.15282 ms_handle_reset on v2:172.21.15.115:6838/1435040687 2024-03-26T18:01:31.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 --2- 172.21.15.49:0/2202404508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cae1e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 -- 172.21.15.49:0/2202404508 >> 172.21.15.49:0/2202404508 conn(0x5624a0c9f030 msgr2=0x5624a0c9e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 -- 172.21.15.49:0/2202404508 shutdown_connections 2024-03-26T18:01:31.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 1 -- 172.21.15.49:0/2202404508 wait complete. 2024-03-26T18:01:31.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.708+0000 7f5fb5421740 20 client.15282 trim_cache size 0 max 16384 2024-03-26T18:01:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.726+0000 7f5fb5421740 1 Processor -- start 2024-03-26T18:01:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.726+0000 7f5fb5421740 1 -- start start 2024-03-26T18:01:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.726+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cb9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.726+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 0x5624a0ca89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.726+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca57d0 0x5624a0ca8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.726+0000 7f5fb5421740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5624a0cba590 con 0x5624a0ca3c40 2024-03-26T18:01:31.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.726+0000 7f5fb5421740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5624a0cbac60 con 0x5624a0ca57d0 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.726+0000 7f5fb5421740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5624a0cbb330 con 0x5624a0ca4e10 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cb9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cb9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60334/0 (socket says 172.21.15.49:60334) 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fabfff640 1 -- 172.21.15.49:0/2872263046 learned_addr learned my addr 172.21.15.49:0/2872263046 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fb0dda640 1 --2- 172.21.15.49:0/2872263046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca57d0 0x5624a0ca8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/2872263046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 0x5624a0ca89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fabfff640 1 -- 172.21.15.49:0/2872263046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 msgr2=0x5624a0ca89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fabfff640 1 --2- 172.21.15.49:0/2872263046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 0x5624a0ca89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fabfff640 1 -- 172.21.15.49:0/2872263046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca57d0 msgr2=0x5624a0ca8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fabfff640 1 --2- 172.21.15.49:0/2872263046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca57d0 0x5624a0ca8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fabfff640 1 -- 172.21.15.49:0/2872263046 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624a0ca4630 con 0x5624a0ca3c40 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/2872263046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 0x5624a0ca89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:31.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5fabfff640 1 --2- 172.21.15.49:0/2872263046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cb9d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c004810 tx=0x7f5f9c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de2fba04a624b64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5f91ffb640 1 -- 172.21.15.49:0/2872263046 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c01e070 con 0x5624a0ca3c40 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5f91ffb640 1 -- 172.21.15.49:0/2872263046 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f9c027070 con 0x5624a0ca3c40 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.727+0000 7f5f91ffb640 1 -- 172.21.15.49:0/2872263046 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c005160 con 0x5624a0ca3c40 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 -- 172.21.15.49:0/2872263046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 msgr2=0x5624a0cb9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 --2- 172.21.15.49:0/2872263046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cb9d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c004810 tx=0x7f5f9c005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 -- 172.21.15.49:0/2872263046 shutdown_connections 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 --2- 172.21.15.49:0/2872263046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca4e10 0x5624a0ca89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 --2- 172.21.15.49:0/2872263046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca57d0 0x5624a0ca8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 --2- 172.21.15.49:0/2872263046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca3c40 0x5624a0cb9d30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 -- 172.21.15.49:0/2872263046 >> 172.21.15.49:0/2872263046 conn(0x5624a0c9f030 msgr2=0x5624a0c9f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 -- 172.21.15.49:0/2872263046 shutdown_connections 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 -- 172.21.15.49:0/2872263046 wait complete. 2024-03-26T18:01:31.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 -- start start 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.728+0000 7f5fb5421740 1 Processor -- start 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0d1eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0d21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fb5421740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca57d0 0x5624a0d235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fb5421740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5624a0cbc0d0 con 0x5624a0ca4e10 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fb5421740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5624a0cbba00 con 0x5624a0ca3c40 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fb5421740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5624a0cbc7a0 con 0x5624a0ca57d0 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fb5421740 10 client.0 _ll_register_callbacks cb 0x7f5f9c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0d21090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0d1eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0d21090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60346/0 (socket says 172.21.15.49:60346) 2024-03-26T18:01:31.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fab7fe640 1 -- 172.21.15.49:0/3700636240 learned_addr learned my addr 172.21.15.49:0/3700636240 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0d1eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:38516/0 (socket says 172.21.15.49:38516) 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fb0dda640 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca57d0 0x5624a0d235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fab7fe640 1 -- 172.21.15.49:0/3700636240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca57d0 msgr2=0x5624a0d235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca57d0 0x5624a0d235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fab7fe640 1 -- 172.21.15.49:0/3700636240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 msgr2=0x5624a0d1eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0d1eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fab7fe640 1 -- 172.21.15.49:0/3700636240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624a0d23b20 con 0x5624a0ca4e10 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fabfff640 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0d1eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fab7fe640 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0d21090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00097e0 tx=0x7f5fa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9573155517310e69 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5f8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/3700636240 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0031070 con 0x5624a0ca4e10 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/3700636240 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f5fa0004020 con 0x5624a0ca4e10 2024-03-26T18:01:31.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.729+0000 7f5fb5421740 1 -- 172.21.15.49:0/3700636240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f9c027070 con 0x5624a0ca4e10 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/3700636240 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fa003c050 con 0x5624a0ca4e10 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5fb5421740 10 client.15288 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5fb5421740 1 -- 172.21.15.49:0/3700636240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624a0d24d60 con 0x5624a0ca4e10 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/3700636240 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f9c027070 con 0x5624a0ca4e10 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/3700636240 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5fa001d070 con 0x5624a0ca4e10 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5fb5421740 10 client.15288 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5fb5421740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5fb5421740 20 client.15288 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5fb5421740 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5624a0d29970 0x5624a0d28680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5fb5421740 4 client.15288 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5fb5421740 1 -- 172.21.15.49:0/3700636240 --> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] -- command(tid 0: {"prefix": "status"}) -- 0x5624a0c0fb00 con 0x5624a0d29970 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.730+0000 7f5fabfff640 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5624a0d29970 0x5624a0d28680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:31.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fabfff640 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5624a0d29970 0x5624a0d28680 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef1762cc14907a7 server_cookie=2e949c4582ed6fab in_seq=0 out_seq=0 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5f8b7fe640 10 client.15288 ms_handle_connect on v2:172.21.15.115:6838/1435040687 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5f8b7fe640 1 -- 172.21.15.49:0/3700636240 <== mds.0 v2:172.21.15.115:6838/1435040687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5624a0c0fb00 con 0x5624a0d29970 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5f8b7fe640 10 client.15288 handle_command_reply: tid=0 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 client.15288 shutdown 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 -- 172.21.15.49:0/3700636240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 msgr2=0x5624a0d21090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0d21090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00097e0 tx=0x7f5fa0004730 comp rx=0 tx=0).stop 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 -- 172.21.15.49:0/3700636240 shutdown_connections 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5624a0d29970 0x5624a0d28680 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624a0ca57d0 0x5624a0d235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5624a0ca4e10 0x5624a0d21090 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5f8b7fe640 0 client.15288 ms_handle_reset on v2:172.21.15.115:6838/1435040687 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 --2- 172.21.15.49:0/3700636240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5624a0ca3c40 0x5624a0d1eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 -- 172.21.15.49:0/3700636240 >> 172.21.15.49:0/3700636240 conn(0x5624a0c9f030 msgr2=0x5624a0cb9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 -- 172.21.15.49:0/3700636240 shutdown_connections 2024-03-26T18:01:31.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 1 -- 172.21.15.49:0/3700636240 wait complete. 2024-03-26T18:01:31.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.731+0000 7f5fb5421740 20 client.15288 trim_cache size 0 max 16384 2024-03-26T18:01:31.721 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:01:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T18:01:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-03-26T18:01:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24533, 2024-03-26T18:01:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T18:01:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T18:01:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T18:01:31.722 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 104.302974183, 2024-03-26T18:01:31.722 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2024-03-26T18:01:31.722 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 92, 2024-03-26T18:01:31.722 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T18:01:31.722 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 108.430012365 2024-03-26T18:01:31.722 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:01:31.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 -- 172.21.15.49:0/1695388402 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5f940a2f20 msgr2=0x7f5f940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5f940a2f20 0x7f5f940a53e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x5624a0ca4d10 tx=0x7f5fa0005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:31.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 -- 172.21.15.49:0/1695388402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fac103480 msgr2=0x7f5fac19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:31.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fac103480 0x7f5fac19f120 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402c630 tx=0x7f5fa4032f20 comp rx=0 tx=0).stop 2024-03-26T18:01:31.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 -- 172.21.15.49:0/1695388402 shutdown_connections 2024-03-26T18:01:31.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5f940a2f20 0x7f5f940a53e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fac102ac0 0x7f5fac1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fac103480 0x7f5fac19f120 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 --2- 172.21.15.49:0/1695388402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5fac1018d0 0x7f5fac197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:31.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 -- 172.21.15.49:0/1695388402 >> 172.21.15.49:0/1695388402 conn(0x7f5fac0fce60 msgr2=0x7f5fac105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:31.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 -- 172.21.15.49:0/1695388402 shutdown_connections 2024-03-26T18:01:31.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:31.732+0000 7f5fb283d640 1 -- 172.21.15.49:0/1695388402 wait complete. 2024-03-26T18:01:31.725 INFO:tasks.cephadm:Setup complete, yielding 2024-03-26T18:01:31.725 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T18:01:31.737 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T18:01:31.737 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-26T18:01:31.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:31 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2876729543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:31.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:31 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1519262994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:31.985 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:32.368 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:32 smithi049 ceph-mon[32331]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:01:32.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:32 smithi171 ceph-mon[36033]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:01:32.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:32 smithi115 ceph-mon[36174]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:01:33.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.787+0000 7fe77c3bc700 1 -- 172.21.15.49:0/1135911962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7741012e0 msgr2=0x7fe774101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.787+0000 7fe77c3bc700 1 --2- 172.21.15.49:0/1135911962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7741012e0 0x7fe774101700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe7700099c0 tx=0x7fe770009cd0 comp rx=0 tx=0).stop 2024-03-26T18:01:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.787+0000 7fe77c3bc700 1 -- 172.21.15.49:0/1135911962 shutdown_connections 2024-03-26T18:01:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.787+0000 7fe77c3bc700 1 --2- 172.21.15.49:0/1135911962 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe774102e80 0x7fe774109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.787+0000 7fe77c3bc700 1 --2- 172.21.15.49:0/1135911962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7741012e0 0x7fe774101700 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.787+0000 7fe77c3bc700 1 --2- 172.21.15.49:0/1135911962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7741024e0 0x7fe774102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.787+0000 7fe77c3bc700 1 -- 172.21.15.49:0/1135911962 >> 172.21.15.49:0/1135911962 conn(0x7fe7740fc880 msgr2=0x7fe7740fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.787+0000 7fe77c3bc700 1 -- 172.21.15.49:0/1135911962 shutdown_connections 2024-03-26T18:01:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.787+0000 7fe77c3bc700 1 -- 172.21.15.49:0/1135911962 wait complete. 2024-03-26T18:01:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77c3bc700 1 Processor -- start 2024-03-26T18:01:33.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77c3bc700 1 -- start start 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77c3bc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7741012e0 0x7fe7741053c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77c3bc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7741024e0 0x7fe774103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77c3bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe774102e80 0x7fe774103f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77c3bc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe77410cec0 con 0x7fe774102e80 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77c3bc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe77410cd40 con 0x7fe7741024e0 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77c3bc700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe77410d040 con 0x7fe7741012e0 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77a959700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe774102e80 0x7fe774103f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77a158700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7741012e0 0x7fe7741053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77a959700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe774102e80 0x7fe774103f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60362/0 (socket says 172.21.15.49:60362) 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe77a959700 1 -- 172.21.15.49:0/2173689094 learned_addr learned my addr 172.21.15.49:0/2173689094 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.788+0000 7fe779957700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7741024e0 0x7fe774103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe779957700 1 -- 172.21.15.49:0/2173689094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7741012e0 msgr2=0x7fe7741053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe779957700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7741012e0 0x7fe7741053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:33.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe779957700 1 -- 172.21.15.49:0/2173689094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe774102e80 msgr2=0x7fe774103f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:33.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe779957700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe774102e80 0x7fe774103f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:33.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe779957700 1 -- 172.21.15.49:0/2173689094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe774104800 con 0x7fe7741024e0 2024-03-26T18:01:33.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe779957700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7741024e0 0x7fe774103a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e9f0 tx=0x7fe76400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:33.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe76b7fe700 1 -- 172.21.15.49:0/2173689094 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76400cb70 con 0x7fe7741024e0 2024-03-26T18:01:33.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe77c3bc700 1 -- 172.21.15.49:0/2173689094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741a27c0 con 0x7fe7741024e0 2024-03-26T18:01:33.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe76b7fe700 1 -- 172.21.15.49:0/2173689094 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76400ee90 con 0x7fe7741024e0 2024-03-26T18:01:33.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe76b7fe700 1 -- 172.21.15.49:0/2173689094 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764003dc0 con 0x7fe7741024e0 2024-03-26T18:01:33.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.789+0000 7fe77c3bc700 1 -- 172.21.15.49:0/2173689094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7741a2cd0 con 0x7fe7741024e0 2024-03-26T18:01:33.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.790+0000 7fe77c3bc700 1 -- 172.21.15.49:0/2173689094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe774069070 con 0x7fe7741024e0 2024-03-26T18:01:33.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.791+0000 7fe76b7fe700 1 -- 172.21.15.49:0/2173689094 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fe764043af0 con 0x7fe7741024e0 2024-03-26T18:01:33.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.792+0000 7fe76b7fe700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe7600a2fb0 0x7fe7600a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:33.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.792+0000 7fe77a158700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe7600a2fb0 0x7fe7600a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:33.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.792+0000 7fe76b7fe700 1 -- 172.21.15.49:0/2173689094 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fe7640cf850 con 0x7fe7741024e0 2024-03-26T18:01:33.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.792+0000 7fe77a158700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe7600a2fb0 0x7fe7600a5470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe770000c00 tx=0x7fe770005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:33.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:33.795+0000 7fe76b7fe700 1 -- 172.21.15.49:0/2173689094 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe76409c340 con 0x7fe7741024e0 2024-03-26T18:01:34.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.018+0000 7fe77c3bc700 1 -- 172.21.15.49:0/2173689094 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fe7741a2f90 con 0x7fe7600a2fb0 2024-03-26T18:01:34.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.020+0000 7fe76b7fe700 1 -- 172.21.15.49:0/2173689094 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fe7741a2f90 con 0x7fe7600a2fb0 2024-03-26T18:01:34.001 INFO:teuthology.orchestra.run.smithi049.stdout:Backend: cephadm 2024-03-26T18:01:34.001 INFO:teuthology.orchestra.run.smithi049.stdout:Available: Yes 2024-03-26T18:01:34.001 INFO:teuthology.orchestra.run.smithi049.stdout:Paused: No 2024-03-26T18:01:34.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 -- 172.21.15.49:0/2173689094 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe7600a2fb0 msgr2=0x7fe7600a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:34.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe7600a2fb0 0x7fe7600a5470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe770000c00 tx=0x7fe770005fb0 comp rx=0 tx=0).stop 2024-03-26T18:01:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 -- 172.21.15.49:0/2173689094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7741024e0 msgr2=0x7fe774103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7741024e0 0x7fe774103a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e9f0 tx=0x7fe76400ed00 comp rx=0 tx=0).stop 2024-03-26T18:01:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 -- 172.21.15.49:0/2173689094 shutdown_connections 2024-03-26T18:01:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fe7600a2fb0 0x7fe7600a5470 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe774102e80 0x7fe774103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7741012e0 0x7fe7741053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 --2- 172.21.15.49:0/2173689094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7741024e0 0x7fe774103a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 -- 172.21.15.49:0/2173689094 >> 172.21.15.49:0/2173689094 conn(0x7fe7740fc880 msgr2=0x7fe77406a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 -- 172.21.15.49:0/2173689094 shutdown_connections 2024-03-26T18:01:34.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:34.022+0000 7fe77c3bc700 1 -- 172.21.15.49:0/2173689094 wait complete. 2024-03-26T18:01:34.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:34 smithi171 ceph-mon[36033]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:34.733 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:34 smithi049 ceph-mon[32331]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:34.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:34 smithi115 ceph-mon[36174]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:34.768 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-26T18:01:35.028 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:35.454 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:35 smithi049 ceph-mon[32331]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:35.606 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:35 smithi171 ceph-mon[36033]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:35.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:35 smithi115 ceph-mon[36174]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:36.472 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:36 smithi049 ceph-mon[32331]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-26T18:01:36.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:36 smithi171 ceph-mon[36033]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-26T18:01:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 -- 172.21.15.49:0/724426115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44e0100c50 msgr2=0x7f44e0103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 --2- 172.21.15.49:0/724426115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44e0100c50 0x7f44e0103080 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009990 tx=0x7f44dc009ca0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 -- 172.21.15.49:0/724426115 shutdown_connections 2024-03-26T18:01:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 --2- 172.21.15.49:0/724426115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e0105f90 0x7f44e0106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 --2- 172.21.15.49:0/724426115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44e0100c50 0x7f44e0103080 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 --2- 172.21.15.49:0/724426115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44e01035c0 0x7f44e0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 -- 172.21.15.49:0/724426115 >> 172.21.15.49:0/724426115 conn(0x7f44e00fc870 msgr2=0x7f44e00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 -- 172.21.15.49:0/724426115 shutdown_connections 2024-03-26T18:01:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 -- 172.21.15.49:0/724426115 wait complete. 2024-03-26T18:01:36.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 Processor -- start 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.703+0000 7f44e7e80700 1 -- start start 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e7e80700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e0100c50 0x7f44e019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e7e80700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44e01035c0 0x7f44e019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e7e80700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44e0105f90 0x7f44e01a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e7e80700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44e0114530 con 0x7f44e01035c0 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e7e80700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f44e01143b0 con 0x7f44e0100c50 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e7e80700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f44e01146b0 con 0x7f44e0105f90 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e541b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44e01035c0 0x7f44e019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e541b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44e01035c0 0x7f44e019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:60366/0 (socket says 172.21.15.49:60366) 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e5c1c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e0100c50 0x7f44e019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e541b700 1 -- 172.21.15.49:0/2594594054 learned_addr learned my addr 172.21.15.49:0/2594594054 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e641d700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44e0105f90 0x7f44e01a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e541b700 1 -- 172.21.15.49:0/2594594054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44e0105f90 msgr2=0x7f44e01a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e541b700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44e0105f90 0x7f44e01a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e541b700 1 -- 172.21.15.49:0/2594594054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e0100c50 msgr2=0x7f44e019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e541b700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e0100c50 0x7f44e019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e541b700 1 -- 172.21.15.49:0/2594594054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e01a67a0 con 0x7f44e01035c0 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e5c1c700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e0100c50 0x7f44e019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e541b700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44e01035c0 0x7f44e019f120 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f44d000e9f0 tx=0x7f44d000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44d6ffd700 1 -- 172.21.15.49:0/2594594054 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d000cbd0 con 0x7f44e01035c0 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44d6ffd700 1 -- 172.21.15.49:0/2594594054 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d000ee90 con 0x7f44e01035c0 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44e7e80700 1 -- 172.21.15.49:0/2594594054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e01a6a90 con 0x7f44e01035c0 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.704+0000 7f44d6ffd700 1 -- 172.21.15.49:0/2594594054 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d00056c0 con 0x7f44e01035c0 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.705+0000 7f44e7e80700 1 -- 172.21.15.49:0/2594594054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e01a6f80 con 0x7f44e01035c0 2024-03-26T18:01:36.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.706+0000 7f44e7e80700 1 -- 172.21.15.49:0/2594594054 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e0069070 con 0x7f44e01035c0 2024-03-26T18:01:36.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.707+0000 7f44d6ffd700 1 -- 172.21.15.49:0/2594594054 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f44d0004750 con 0x7f44e01035c0 2024-03-26T18:01:36.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.707+0000 7f44d6ffd700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f44cc0a2fb0 0x7f44cc0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.707+0000 7f44d6ffd700 1 -- 172.21.15.49:0/2594594054 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f44d00cf990 con 0x7f44e01035c0 2024-03-26T18:01:36.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.707+0000 7f44e5c1c700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f44cc0a2fb0 0x7f44cc0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.708+0000 7f44e5c1c700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f44cc0a2fb0 0x7f44cc0a5470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00b2e0 tx=0x7f44dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.712+0000 7f44d6ffd700 1 -- 172.21.15.49:0/2594594054 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f44d009c480 con 0x7f44e01035c0 2024-03-26T18:01:36.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:36 smithi115 ceph-mon[36174]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-26T18:01:36.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.927+0000 7f44e7e80700 1 -- 172.21.15.49:0/2594594054 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f44e00627f0 con 0x7f44cc0a2fb0 2024-03-26T18:01:36.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.941+0000 7f44d6ffd700 1 -- 172.21.15.49:0/2594594054 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f44e00627f0 con 0x7f44cc0a2fb0 2024-03-26T18:01:36.922 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-26T18:01:36.922 INFO:teuthology.orchestra.run.smithi049.stdout:mds.a smithi049 running (117s) 19s ago 116s 17.6M - 19.0.0-2311-g36051606 497bb87bc5da e76a9c150d1b 2024-03-26T18:01:36.922 INFO:teuthology.orchestra.run.smithi049.stdout:mds.b smithi115 running (114s) 47s ago 113s 26.5M - 19.0.0-2311-g36051606 497bb87bc5da ab1506df1fcc 2024-03-26T18:01:36.922 INFO:teuthology.orchestra.run.smithi049.stdout:mds.c smithi171 running (2m) 47s ago 2m 16.3M - 19.0.0-2311-g36051606 497bb87bc5da 259c053f8c6f 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mds.d smithi049 running (2m) 19s ago 2m 12.9M - 19.0.0-2311-g36051606 497bb87bc5da c008ea39c957 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mds.e smithi115 running (2m) 47s ago 2m 17.3M - 19.0.0-2311-g36051606 497bb87bc5da 82c9270a6458 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mds.f smithi171 running (2m) 47s ago 2m 18.8M - 19.0.0-2311-g36051606 497bb87bc5da 3e7821f4b5e0 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mds.g smithi049 running (119s) 19s ago 119s 16.3M - 19.0.0-2311-g36051606 497bb87bc5da 052ef4e36fe0 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mds.h smithi115 running (2m) 47s ago 2m 17.5M - 19.0.0-2311-g36051606 497bb87bc5da 2420a64b3c8e 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mds.i smithi171 running (2m) 47s ago 2m 17.5M - 19.0.0-2311-g36051606 497bb87bc5da 2f2c017b3885 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mds.j smithi049 running (2m) 19s ago 2m 14.0M - 19.0.0-2311-g36051606 497bb87bc5da c7eceaac0a99 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mds.k smithi115 running (2m) 47s ago 2m 17.4M - 19.0.0-2311-g36051606 497bb87bc5da d6dc72aac720 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mds.l smithi171 running (2m) 47s ago 2m 18.5M - 19.0.0-2311-g36051606 497bb87bc5da b3e097598a09 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.x smithi049 *:9283,8765 running (9m) 19s ago 9m 514M - 19.0.0-2311-g36051606 497bb87bc5da 819b241b3d51 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.y smithi115 *:8443,8765 running (7m) 47s ago 7m 453M - 19.0.0-2311-g36051606 497bb87bc5da e3e1ac33f563 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.z smithi171 *:8443,8765 running (7m) 47s ago 7m 454M - 19.0.0-2311-g36051606 497bb87bc5da ecd495246d5c 2024-03-26T18:01:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:mon.a smithi049 running (9m) 19s ago 9m 52.5M 2048M 19.0.0-2311-g36051606 497bb87bc5da f601912b7bc1 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:mon.b smithi115 running (8m) 47s ago 7m 38.9M 2048M 19.0.0-2311-g36051606 497bb87bc5da e80310a4705b 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:mon.c smithi171 running (8m) 47s ago 8m 40.4M 2048M 19.0.0-2311-g36051606 497bb87bc5da 4f0304e95a28 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (7m) 19s ago 7m 55.3M 4272M 19.0.0-2311-g36051606 497bb87bc5da 0c21864abd9d 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi115 running (6m) 47s ago 6m 47.1M 4270M 19.0.0-2311-g36051606 497bb87bc5da b4373cd2f114 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi171 running (6m) 47s ago 6m 46.9M 4270M 19.0.0-2311-g36051606 497bb87bc5da b5605db6d644 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi049 running (6m) 19s ago 6m 54.2M 4272M 19.0.0-2311-g36051606 497bb87bc5da 69e58e37b585 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi115 running (5m) 47s ago 5m 42.9M 4270M 19.0.0-2311-g36051606 497bb87bc5da 6078ddadc075 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi171 running (5m) 47s ago 5m 44.7M 4270M 19.0.0-2311-g36051606 497bb87bc5da e6a09bd763d1 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 smithi049 running (4m) 19s ago 4m 45.3M 4272M 19.0.0-2311-g36051606 497bb87bc5da 00681a64db7e 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 smithi115 running (4m) 47s ago 4m 56.0M 4270M 19.0.0-2311-g36051606 497bb87bc5da e1a313ce9524 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.8 smithi171 running (4m) 47s ago 4m 42.7M 4270M 19.0.0-2311-g36051606 497bb87bc5da 006fdd7ab605 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.9 smithi049 running (3m) 19s ago 3m 42.1M 4272M 19.0.0-2311-g36051606 497bb87bc5da 5851a079df2f 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.10 smithi115 running (3m) 47s ago 3m 54.1M 4270M 19.0.0-2311-g36051606 497bb87bc5da 913fad27d5a1 2024-03-26T18:01:36.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.11 smithi171 running (2m) 47s ago 2m 43.0M 4270M 19.0.0-2311-g36051606 497bb87bc5da 8a2aa97ad59b 2024-03-26T18:01:36.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 -- 172.21.15.49:0/2594594054 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f44cc0a2fb0 msgr2=0x7f44cc0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f44cc0a2fb0 0x7f44cc0a5470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00b2e0 tx=0x7f44dc005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 -- 172.21.15.49:0/2594594054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44e01035c0 msgr2=0x7f44e019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44e01035c0 0x7f44e019f120 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f44d000e9f0 tx=0x7f44d000ed00 comp rx=0 tx=0).stop 2024-03-26T18:01:36.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 -- 172.21.15.49:0/2594594054 shutdown_connections 2024-03-26T18:01:36.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f44cc0a2fb0 0x7f44cc0a5470 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44e0105f90 0x7f44e01a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44e0100c50 0x7f44e019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 --2- 172.21.15.49:0/2594594054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f44e01035c0 0x7f44e019f120 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 -- 172.21.15.49:0/2594594054 >> 172.21.15.49:0/2594594054 conn(0x7f44e00fc870 msgr2=0x7f44e0104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 -- 172.21.15.49:0/2594594054 shutdown_connections 2024-03-26T18:01:36.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:36.943+0000 7f44e7e80700 1 -- 172.21.15.49:0/2594594054 wait complete. 2024-03-26T18:01:37.757 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-26T18:01:38.017 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:38.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:38 smithi115 ceph-mon[36174]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-26T18:01:38.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:38 smithi115 ceph-mon[36174]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:38.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:38 smithi049 ceph-mon[32331]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-26T18:01:38.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:38 smithi049 ceph-mon[32331]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:38.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:38 smithi171 ceph-mon[36033]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-26T18:01:38.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:38 smithi171 ceph-mon[36033]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.872+0000 7efc4b8c6700 1 -- 172.21.15.49:0/1057845464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc441012e0 msgr2=0x7efc44101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.872+0000 7efc4b8c6700 1 --2- 172.21.15.49:0/1057845464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc441012e0 0x7efc44101700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efc3400b0d0 tx=0x7efc3400b3e0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.872+0000 7efc4b8c6700 1 -- 172.21.15.49:0/1057845464 shutdown_connections 2024-03-26T18:01:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.872+0000 7efc4b8c6700 1 --2- 172.21.15.49:0/1057845464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc44102e80 0x7efc44109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.872+0000 7efc4b8c6700 1 --2- 172.21.15.49:0/1057845464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc441012e0 0x7efc44101700 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.872+0000 7efc4b8c6700 1 --2- 172.21.15.49:0/1057845464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc441024e0 0x7efc44102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.872+0000 7efc4b8c6700 1 -- 172.21.15.49:0/1057845464 >> 172.21.15.49:0/1057845464 conn(0x7efc440fc880 msgr2=0x7efc440fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.872+0000 7efc4b8c6700 1 -- 172.21.15.49:0/1057845464 shutdown_connections 2024-03-26T18:01:39.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.872+0000 7efc4b8c6700 1 -- 172.21.15.49:0/1057845464 wait complete. 2024-03-26T18:01:39.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc4b8c6700 1 Processor -- start 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc4b8c6700 1 -- start start 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc4b8c6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc441012e0 0x7efc441975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc4b8c6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc441024e0 0x7efc44197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc4b8c6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc44102e80 0x7efc4419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc4b8c6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efc4410cf20 con 0x7efc441012e0 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc4b8c6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7efc4410cda0 con 0x7efc44102e80 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc4b8c6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efc4410d0a0 con 0x7efc441024e0 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc49662700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc441012e0 0x7efc441975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc49662700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc441012e0 0x7efc441975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52174/0 (socket says 172.21.15.49:52174) 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc49662700 1 -- 172.21.15.49:0/2463300708 learned_addr learned my addr 172.21.15.49:0/2463300708 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc49e63700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc44102e80 0x7efc4419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.873+0000 7efc48e61700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc441024e0 0x7efc44197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc49662700 1 -- 172.21.15.49:0/2463300708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc441024e0 msgr2=0x7efc44197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc49662700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc441024e0 0x7efc44197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc49662700 1 -- 172.21.15.49:0/2463300708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc44102e80 msgr2=0x7efc4419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc49662700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc44102e80 0x7efc4419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc49662700 1 -- 172.21.15.49:0/2463300708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4419f210 con 0x7efc441012e0 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc49662700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc441012e0 0x7efc441975b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efc34007a20 tx=0x7efc34007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc427fc700 1 -- 172.21.15.49:0/2463300708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34003c00 con 0x7efc441012e0 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc427fc700 1 -- 172.21.15.49:0/2463300708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3401e070 con 0x7efc441012e0 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc427fc700 1 -- 172.21.15.49:0/2463300708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc340053d0 con 0x7efc441012e0 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc4b8c6700 1 -- 172.21.15.49:0/2463300708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4406a060 con 0x7efc441012e0 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.874+0000 7efc4b8c6700 1 -- 172.21.15.49:0/2463300708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4406a550 con 0x7efc441012e0 2024-03-26T18:01:39.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.875+0000 7efc4b8c6700 1 -- 172.21.15.49:0/2463300708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4406af20 con 0x7efc441012e0 2024-03-26T18:01:39.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.876+0000 7efc427fc700 1 -- 172.21.15.49:0/2463300708 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7efc3401aa30 con 0x7efc441012e0 2024-03-26T18:01:39.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.877+0000 7efc427fc700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc2c0a3030 0x7efc2c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.877+0000 7efc48e61700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc2c0a3030 0x7efc2c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.877+0000 7efc427fc700 1 -- 172.21.15.49:0/2463300708 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7efc3400f020 con 0x7efc441012e0 2024-03-26T18:01:39.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.877+0000 7efc48e61700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc2c0a3030 0x7efc2c0a54f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efc38009c60 tx=0x7efc38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:39.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:39.880+0000 7efc427fc700 1 -- 172.21.15.49:0/2463300708 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efc3402a020 con 0x7efc441012e0 2024-03-26T18:01:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.094+0000 7efc4b8c6700 1 -- 172.21.15.49:0/2463300708 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7efc4406a720 con 0x7efc2c0a3030 2024-03-26T18:01:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.100+0000 7efc427fc700 1 -- 172.21.15.49:0/2463300708 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7efc4406a720 con 0x7efc2c0a3030 2024-03-26T18:01:40.082 INFO:teuthology.orchestra.run.smithi049.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-26T18:01:40.082 INFO:teuthology.orchestra.run.smithi049.stdout:mds.all 12/12 50s ago 2m smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-03-26T18:01:40.082 INFO:teuthology.orchestra.run.smithi049.stdout:mgr 3/3 50s ago 7m smithi049=x;smithi115=y;smithi171=z;count:3 2024-03-26T18:01:40.082 INFO:teuthology.orchestra.run.smithi049.stdout:mon 3/3 50s ago 8m smithi049:172.21.15.49=a;smithi115:172.21.15.115=b;smithi171:172.21.15.171=c;count:3 2024-03-26T18:01:40.082 INFO:teuthology.orchestra.run.smithi049.stdout:osd 12 50s ago - 2024-03-26T18:01:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 -- 172.21.15.49:0/2463300708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc2c0a3030 msgr2=0x7efc2c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc2c0a3030 0x7efc2c0a54f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efc38009c60 tx=0x7efc38009260 comp rx=0 tx=0).stop 2024-03-26T18:01:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 -- 172.21.15.49:0/2463300708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc441012e0 msgr2=0x7efc441975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc441012e0 0x7efc441975b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efc34007a20 tx=0x7efc34007a50 comp rx=0 tx=0).stop 2024-03-26T18:01:40.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 -- 172.21.15.49:0/2463300708 shutdown_connections 2024-03-26T18:01:40.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7efc2c0a3030 0x7efc2c0a54f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc44102e80 0x7efc4419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc441012e0 0x7efc441975b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 --2- 172.21.15.49:0/2463300708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc441024e0 0x7efc44197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 -- 172.21.15.49:0/2463300708 >> 172.21.15.49:0/2463300708 conn(0x7efc440fc880 msgr2=0x7efc441058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:40.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 -- 172.21.15.49:0/2463300708 shutdown_connections 2024-03-26T18:01:40.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:40.103+0000 7efc4b8c6700 1 -- 172.21.15.49:0/2463300708 wait complete. 2024-03-26T18:01:40.714 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:40 smithi049 ceph-mon[32331]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:40.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:40 smithi115 ceph-mon[36174]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:40.739 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-26T18:01:40.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:40 smithi171 ceph-mon[36033]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:40.992 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:41.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:41 smithi115 ceph-mon[36174]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:41.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:41 smithi049 ceph-mon[32331]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:41.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:41 smithi171 ceph-mon[36033]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:42.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:42 smithi049 ceph-mon[32331]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:01:42.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:42 smithi115 ceph-mon[36174]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:01:42.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.824+0000 7f30b5045700 1 -- 172.21.15.49:0/2761235225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b01012d0 msgr2=0x7f30b01016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:42.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.824+0000 7f30b5045700 1 --2- 172.21.15.49:0/2761235225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b01012d0 0x7f30b01016f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f30a800b0d0 tx=0x7f30a800b3e0 comp rx=0 tx=0).stop 2024-03-26T18:01:42.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.824+0000 7f30b5045700 1 -- 172.21.15.49:0/2761235225 shutdown_connections 2024-03-26T18:01:42.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.824+0000 7f30b5045700 1 --2- 172.21.15.49:0/2761235225 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30b0102e70 0x7f30b0109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:42.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.824+0000 7f30b5045700 1 --2- 172.21.15.49:0/2761235225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b01012d0 0x7f30b01016f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:42.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.824+0000 7f30b5045700 1 --2- 172.21.15.49:0/2761235225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b01024d0 0x7f30b0102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:42.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.824+0000 7f30b5045700 1 -- 172.21.15.49:0/2761235225 >> 172.21.15.49:0/2761235225 conn(0x7f30b00fc870 msgr2=0x7f30b00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:42.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.824+0000 7f30b5045700 1 -- 172.21.15.49:0/2761235225 shutdown_connections 2024-03-26T18:01:42.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.824+0000 7f30b5045700 1 -- 172.21.15.49:0/2761235225 wait complete. 2024-03-26T18:01:42.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.824+0000 7f30b5045700 1 Processor -- start 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30b5045700 1 -- start start 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30b5045700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b01012d0 0x7f30b019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30b5045700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b01024d0 0x7f30b019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30b5045700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30b0102e70 0x7f30b01a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30b5045700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f30b010ceb0 con 0x7f30b01024d0 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30b5045700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f30b010cd30 con 0x7f30b01012d0 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30b5045700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f30b010d030 con 0x7f30b0102e70 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30aed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b01012d0 0x7f30b019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30ae59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b01024d0 0x7f30b019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30aed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b01012d0 0x7f30b019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:46824/0 (socket says 172.21.15.49:46824) 2024-03-26T18:01:42.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30aed9d700 1 -- 172.21.15.49:0/2858707527 learned_addr learned my addr 172.21.15.49:0/2858707527 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30af59e700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30b0102e70 0x7f30b01a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30aed9d700 1 -- 172.21.15.49:0/2858707527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30b0102e70 msgr2=0x7f30b01a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30aed9d700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30b0102e70 0x7f30b01a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30aed9d700 1 -- 172.21.15.49:0/2858707527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b01024d0 msgr2=0x7f30b019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30aed9d700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b01024d0 0x7f30b019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.825+0000 7f30aed9d700 1 -- 172.21.15.49:0/2858707527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b01a2450 con 0x7f30b01012d0 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.826+0000 7f30aed9d700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b01012d0 0x7f30b019a7f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f30a800b770 tx=0x7f30a8009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.826+0000 7f30a7fff700 1 -- 172.21.15.49:0/2858707527 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8003c00 con 0x7f30b01012d0 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.826+0000 7f30a7fff700 1 -- 172.21.15.49:0/2858707527 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a8022070 con 0x7f30b01012d0 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.826+0000 7f30a7fff700 1 -- 172.21.15.49:0/2858707527 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a801b7e0 con 0x7f30b01012d0 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.826+0000 7f30b5045700 1 -- 172.21.15.49:0/2858707527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b01a26e0 con 0x7f30b01012d0 2024-03-26T18:01:42.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.826+0000 7f30b5045700 1 -- 172.21.15.49:0/2858707527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b01a2b30 con 0x7f30b01012d0 2024-03-26T18:01:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.827+0000 7f30b5045700 1 -- 172.21.15.49:0/2858707527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b0069070 con 0x7f30b01012d0 2024-03-26T18:01:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.828+0000 7f30a7fff700 1 -- 172.21.15.49:0/2858707527 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f30a801da70 con 0x7f30b01012d0 2024-03-26T18:01:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.828+0000 7f30a7fff700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f309c0a2fb0 0x7f309c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:42.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.829+0000 7f30a7fff700 1 -- 172.21.15.49:0/2858707527 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f30a800f020 con 0x7f30b01012d0 2024-03-26T18:01:42.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.829+0000 7f30ae59c700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f309c0a2fb0 0x7f309c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:42.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.829+0000 7f30ae59c700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f309c0a2fb0 0x7f309c0a5470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3098009c60 tx=0x7f3098009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:42.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:42.833+0000 7f30a7fff700 1 -- 172.21.15.49:0/2858707527 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f30a809d130 con 0x7f30b01012d0 2024-03-26T18:01:42.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:42 smithi171 ceph-mon[36033]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-26T18:01:43.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.048+0000 7f30b5045700 1 -- 172.21.15.49:0/2858707527 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f30b00fea00 con 0x7f309c0a2fb0 2024-03-26T18:01:43.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.050+0000 7f30a7fff700 1 -- 172.21.15.49:0/2858707527 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f30b00fea00 con 0x7f309c0a2fb0 2024-03-26T18:01:43.031 INFO:teuthology.orchestra.run.smithi049.stdout:HOST ADDR LABELS STATUS 2024-03-26T18:01:43.031 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 172.21.15.49 2024-03-26T18:01:43.031 INFO:teuthology.orchestra.run.smithi049.stdout:smithi115 172.21.15.115 2024-03-26T18:01:43.032 INFO:teuthology.orchestra.run.smithi049.stdout:smithi171 172.21.15.171 2024-03-26T18:01:43.032 INFO:teuthology.orchestra.run.smithi049.stdout:3 hosts in cluster 2024-03-26T18:01:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.052+0000 7f30b5045700 1 -- 172.21.15.49:0/2858707527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f309c0a2fb0 msgr2=0x7f309c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.052+0000 7f30b5045700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f309c0a2fb0 0x7f309c0a5470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3098009c60 tx=0x7f3098009260 comp rx=0 tx=0).stop 2024-03-26T18:01:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.052+0000 7f30b5045700 1 -- 172.21.15.49:0/2858707527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b01012d0 msgr2=0x7f30b019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.052+0000 7f30b5045700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b01012d0 0x7f30b019a7f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f30a800b770 tx=0x7f30a8009810 comp rx=0 tx=0).stop 2024-03-26T18:01:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.052+0000 7f30b5045700 1 -- 172.21.15.49:0/2858707527 shutdown_connections 2024-03-26T18:01:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.052+0000 7f30b5045700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f309c0a2fb0 0x7f309c0a5470 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.052+0000 7f30b5045700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30b0102e70 0x7f30b01a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.052+0000 7f30b5045700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30b01012d0 0x7f30b019a7f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.052+0000 7f30b5045700 1 --2- 172.21.15.49:0/2858707527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30b01024d0 0x7f30b019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:43.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.052+0000 7f30b5045700 1 -- 172.21.15.49:0/2858707527 >> 172.21.15.49:0/2858707527 conn(0x7f30b00fc870 msgr2=0x7f30b00fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:43.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.053+0000 7f30b5045700 1 -- 172.21.15.49:0/2858707527 shutdown_connections 2024-03-26T18:01:43.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:43.053+0000 7f30b5045700 1 -- 172.21.15.49:0/2858707527 wait complete. 2024-03-26T18:01:43.875 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-26T18:01:44.127 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:44.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:44 smithi049 ceph-mon[32331]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:44.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:44 smithi049 ceph-mon[32331]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:44.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-03-26T18:01:44.502 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:44 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2024-03-26T18:01:44.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:44 smithi115 ceph-mon[36174]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:44.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:44 smithi115 ceph-mon[36174]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:44.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-03-26T18:01:44.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:44 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2024-03-26T18:01:44.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:44 smithi171 ceph-mon[36033]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-26T18:01:44.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:44 smithi171 ceph-mon[36033]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:44.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-03-26T18:01:44.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:44 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2024-03-26T18:01:45.552 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-03-26T18:01:45.552 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:45 smithi049 ceph-mon[32331]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2024-03-26T18:01:45.552 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:45 smithi049 ceph-mon[32331]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T18:01:45.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-03-26T18:01:45.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:45 smithi115 ceph-mon[36174]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2024-03-26T18:01:45.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:45 smithi115 ceph-mon[36174]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T18:01:45.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-03-26T18:01:45.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:45 smithi171 ceph-mon[36033]: from='mgr.14152 172.21.15.49:0/1343020450' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2024-03-26T18:01:45.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:45 smithi171 ceph-mon[36033]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T18:01:45.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.952+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3291791910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c8103a50 msgr2=0x7fa7c8103e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:45.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.952+0000 7fa7cdb94700 1 --2- 172.21.15.49:0/3291791910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c8103a50 0x7fa7c8103e70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009990 tx=0x7fa7b0009ca0 comp rx=0 tx=0).stop 2024-03-26T18:01:45.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.952+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3291791910 shutdown_connections 2024-03-26T18:01:45.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.952+0000 7fa7cdb94700 1 --2- 172.21.15.49:0/3291791910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c8100e70 0x7fa7c810aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:45.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.952+0000 7fa7cdb94700 1 --2- 172.21.15.49:0/3291791910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c8103a50 0x7fa7c8103e70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:45.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.952+0000 7fa7cdb94700 1 --2- 172.21.15.49:0/3291791910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7c81004d0 0x7fa7c8100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:45.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.952+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3291791910 >> 172.21.15.49:0/3291791910 conn(0x7fa7c80fc050 msgr2=0x7fa7c80fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:45.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.952+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3291791910 shutdown_connections 2024-03-26T18:01:45.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.952+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3291791910 wait complete. 2024-03-26T18:01:45.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7cdb94700 1 Processor -- start 2024-03-26T18:01:45.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7cdb94700 1 -- start start 2024-03-26T18:01:45.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7cdb94700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7c81004d0 0x7fa7c8103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7cdb94700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c8100e70 0x7fa7c81017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7cdb94700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c8103a50 0x7fa7c8101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7cdb94700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c810e000 con 0x7fa7c8100e70 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7cdb94700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c810de80 con 0x7fa7c8103a50 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7cdb94700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c810e180 con 0x7fa7c81004d0 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7c6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c8100e70 0x7fa7c81017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7c6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c8100e70 0x7fa7c81017a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52194/0 (socket says 172.21.15.49:52194) 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7c7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c8103a50 0x7fa7c8101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7c77fe700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7c81004d0 0x7fa7c8103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.953+0000 7fa7c6ffd700 1 -- 172.21.15.49:0/3719726292 learned_addr learned my addr 172.21.15.49:0/3719726292 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7c6ffd700 1 -- 172.21.15.49:0/3719726292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7c81004d0 msgr2=0x7fa7c8103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7c6ffd700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7c81004d0 0x7fa7c8103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:45.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7c6ffd700 1 -- 172.21.15.49:0/3719726292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c8103a50 msgr2=0x7fa7c8101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7c6ffd700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c8103a50 0x7fa7c8101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7c6ffd700 1 -- 172.21.15.49:0/3719726292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c8102590 con 0x7fa7c8100e70 2024-03-26T18:01:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7c7fff700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c8103a50 0x7fa7c8101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7c6ffd700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c8100e70 0x7fa7c81017a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800e9c0 tx=0x7fa7b800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7c4ff9700 1 -- 172.21.15.49:0/3719726292 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b800cbd0 con 0x7fa7c8100e70 2024-03-26T18:01:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3719726292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c81a2740 con 0x7fa7c8100e70 2024-03-26T18:01:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7c4ff9700 1 -- 172.21.15.49:0/3719726292 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b800ee60 con 0x7fa7c8100e70 2024-03-26T18:01:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7c4ff9700 1 -- 172.21.15.49:0/3719726292 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8005630 con 0x7fa7c8100e70 2024-03-26T18:01:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.954+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3719726292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c81a2c50 con 0x7fa7c8100e70 2024-03-26T18:01:45.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.955+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3719726292 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c8069070 con 0x7fa7c8100e70 2024-03-26T18:01:45.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.957+0000 7fa7c4ff9700 1 -- 172.21.15.49:0/3719726292 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fa7b8004750 con 0x7fa7c8100e70 2024-03-26T18:01:45.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.957+0000 7fa7c4ff9700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa7b40a2fb0 0x7fa7b40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:45.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.957+0000 7fa7c77fe700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa7b40a2fb0 0x7fa7b40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:45.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.957+0000 7fa7c4ff9700 1 -- 172.21.15.49:0/3719726292 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7fa7b80cf9d0 con 0x7fa7c8100e70 2024-03-26T18:01:45.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.958+0000 7fa7c77fe700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa7b40a2fb0 0x7fa7b40a5470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000b7b0 tx=0x7fa7b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:45.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:45.962+0000 7fa7c4ff9700 1 -- 172.21.15.49:0/3719726292 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa7b809c480 con 0x7fa7c8100e70 2024-03-26T18:01:46.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.176+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3719726292 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa7c80627f0 con 0x7fa7b40a2fb0 2024-03-26T18:01:46.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.178+0000 7fa7c4ff9700 1 -- 172.21.15.49:0/3719726292 <== mgr.14152 v2:172.21.15.49:6800/3127469495 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa7c80627f0 con 0x7fa7b40a2fb0 2024-03-26T18:01:46.159 INFO:teuthology.orchestra.run.smithi049.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-26T18:01:46.160 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747500D2480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-26T18:01:46.160 INFO:teuthology.orchestra.run.smithi049.stdout:smithi115 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UU400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-26T18:01:46.160 INFO:teuthology.orchestra.run.smithi049.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-26T18:01:46.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.180+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3719726292 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa7b40a2fb0 msgr2=0x7fa7b40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:46.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.180+0000 7fa7cdb94700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa7b40a2fb0 0x7fa7b40a5470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000b7b0 tx=0x7fa7b0005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:46.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.180+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3719726292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c8100e70 msgr2=0x7fa7c81017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:46.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.180+0000 7fa7cdb94700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c8100e70 0x7fa7c81017a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800e9c0 tx=0x7fa7b800ecd0 comp rx=0 tx=0).stop 2024-03-26T18:01:46.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.180+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3719726292 shutdown_connections 2024-03-26T18:01:46.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.180+0000 7fa7cdb94700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa7b40a2fb0 0x7fa7b40a5470 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:46.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.180+0000 7fa7cdb94700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7c8103a50 0x7fa7c8101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:46.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.180+0000 7fa7cdb94700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7c81004d0 0x7fa7c8103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:46.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.180+0000 7fa7cdb94700 1 --2- 172.21.15.49:0/3719726292 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7c8100e70 0x7fa7c81017a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:46.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.180+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3719726292 >> 172.21.15.49:0/3719726292 conn(0x7fa7c80fc050 msgr2=0x7fa7c8106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:46.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.181+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3719726292 shutdown_connections 2024-03-26T18:01:46.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:46.181+0000 7fa7cdb94700 1 -- 172.21.15.49:0/3719726292 wait complete. 2024-03-26T18:01:46.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:46 smithi115 ceph-mon[36174]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:01:46.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:46 smithi115 ceph-mon[36174]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T18:01:46.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:46 smithi049 ceph-mon[32331]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:01:46.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:46 smithi049 ceph-mon[32331]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T18:01:46.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:46 smithi171 ceph-mon[36033]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:01:46.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:46 smithi171 ceph-mon[36033]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T18:01:46.861 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T18:01:46.873 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T18:01:46.873 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-26T18:01:47.127 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:47.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:47 smithi049 ceph-mon[32331]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:47.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:47 smithi115 ceph-mon[36174]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:47.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:47 smithi171 ceph-mon[36033]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:01:48.681 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:48 smithi049 ceph-mon[32331]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-26T18:01:48.681 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:48 smithi049 ceph-mon[32331]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-26T18:01:48.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:48 smithi115 ceph-mon[36174]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-26T18:01:48.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:48 smithi115 ceph-mon[36174]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-26T18:01:48.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:48 smithi171 ceph-mon[36033]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-03-26T18:01:48.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:48 smithi171 ceph-mon[36033]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-26T18:01:48.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.945+0000 7f679d1f8700 1 -- 172.21.15.49:0/3964526918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798103fa0 msgr2=0x7f679810a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.945+0000 7f679d1f8700 1 --2- 172.21.15.49:0/3964526918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798103fa0 0x7f679810a850 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f67880099c0 tx=0x7f6788009cd0 comp rx=0 tx=0).stop 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.945+0000 7f679d1f8700 1 -- 172.21.15.49:0/3964526918 shutdown_connections 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.945+0000 7f679d1f8700 1 --2- 172.21.15.49:0/3964526918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798103fa0 0x7f679810a850 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.945+0000 7f679d1f8700 1 --2- 172.21.15.49:0/3964526918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6798102400 0x7f6798102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.945+0000 7f679d1f8700 1 --2- 172.21.15.49:0/3964526918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6798103600 0x7f6798103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.945+0000 7f679d1f8700 1 -- 172.21.15.49:0/3964526918 >> 172.21.15.49:0/3964526918 conn(0x7f67980fd9c0 msgr2=0x7f67980ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.945+0000 7f679d1f8700 1 -- 172.21.15.49:0/3964526918 shutdown_connections 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.945+0000 7f679d1f8700 1 -- 172.21.15.49:0/3964526918 wait complete. 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679d1f8700 1 Processor -- start 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679d1f8700 1 -- start start 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679d1f8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6798102400 0x7f6798198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679d1f8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6798103600 0x7f6798198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679d1f8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798103fa0 0x7f679819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:48.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679d1f8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f679810dff0 con 0x7f6798103fa0 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679d1f8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f679810de70 con 0x7f6798103600 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679d1f8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f679810e170 con 0x7f6798102400 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f6796d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6798102400 0x7f6798198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6798103600 0x7f6798198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679759e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798103fa0 0x7f679819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f6796d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6798102400 0x7f6798198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:36402/0 (socket says 172.21.15.49:36402) 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f679659c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6798103600 0x7f6798198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48284/0 (socket says 172.21.15.49:48284) 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.946+0000 7f6796d9d700 1 -- 172.21.15.49:0/2012266954 learned_addr learned my addr 172.21.15.49:0/2012266954 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f6796d9d700 1 -- 172.21.15.49:0/2012266954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6798103600 msgr2=0x7f6798198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f6796d9d700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6798103600 0x7f6798198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f6796d9d700 1 -- 172.21.15.49:0/2012266954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798103fa0 msgr2=0x7f679819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f6796d9d700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798103fa0 0x7f679819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f6796d9d700 1 -- 172.21.15.49:0/2012266954 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67981a01d0 con 0x7f6798102400 2024-03-26T18:01:48.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f679659c700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6798103600 0x7f6798198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:48.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f6796d9d700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6798102400 0x7f6798198570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6790009f20 tx=0x7f679000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:48.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f678ffff700 1 -- 172.21.15.49:0/2012266954 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790003a00 con 0x7f6798102400 2024-03-26T18:01:48.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f679d1f8700 1 -- 172.21.15.49:0/2012266954 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67981a04c0 con 0x7f6798102400 2024-03-26T18:01:48.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f678ffff700 1 -- 172.21.15.49:0/2012266954 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6790003ba0 con 0x7f6798102400 2024-03-26T18:01:48.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f678ffff700 1 -- 172.21.15.49:0/2012266954 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790004890 con 0x7f6798102400 2024-03-26T18:01:48.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.947+0000 7f679d1f8700 1 -- 172.21.15.49:0/2012266954 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67981a0a00 con 0x7f6798102400 2024-03-26T18:01:48.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.948+0000 7f679d1f8700 1 -- 172.21.15.49:0/2012266954 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6798069070 con 0x7f6798102400 2024-03-26T18:01:48.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.949+0000 7f678ffff700 1 -- 172.21.15.49:0/2012266954 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f6790012050 con 0x7f6798102400 2024-03-26T18:01:48.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.950+0000 7f678ffff700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67840a2fb0 0x7f67840a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:48.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.950+0000 7f679659c700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67840a2fb0 0x7f67840a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:48.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.950+0000 7f678ffff700 1 -- 172.21.15.49:0/2012266954 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f67900cf840 con 0x7f6798102400 2024-03-26T18:01:48.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.951+0000 7f679659c700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67840a2fb0 0x7f67840a5470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6780011490 tx=0x7f6780009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:48.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:48.955+0000 7f678ffff700 1 -- 172.21.15.49:0/2012266954 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6790010040 con 0x7f6798102400 2024-03-26T18:01:49.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.212+0000 7f679d1f8700 1 -- 172.21.15.49:0/2012266954 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6798062fc0 con 0x7f6798102400 2024-03-26T18:01:49.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.212+0000 7f678ffff700 1 -- 172.21.15.49:0/2012266954 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3099 (secure 0 0 0) 0x7f679009c2f0 con 0x7f6798102400 2024-03-26T18:01:49.194 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2024-03-26T18:01:49.194 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T18:01:49.194 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 28 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-03-26T17:59:46.796381+0000 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-03-26T18:00:09.450708+0000 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 300 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2024-03-26T18:01:49.195 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 0 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 5 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1,2,3,4 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=24533,1=24503,2=24478,3=24428,4=24388} 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3,4] 2024-03-26T18:01:49.196 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data disabled 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 0 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:qdb_cluster leader: 24533 members: 24478,24503,24388,24428,24533 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b{0:24533} state up:active seq 3 addr [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.e{0:24467} state up:standby-replay seq 1 addr [v2:172.21.15.115:6834/4085026954,v1:172.21.15.115:6835/4085026954] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.h{1:24503} state up:active seq 10 addr [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c{1:24451} state up:standby-replay seq 1 addr [v2:172.21.15.171:6836/2428288879,v1:172.21.15.171:6837/2428288879] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.i{2:24478} state up:active seq 9 addr [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.f{2:24439} state up:standby-replay seq 1 addr [v2:172.21.15.171:6834/3854985421,v1:172.21.15.171:6835/3854985421] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.k{3:24428} state up:active seq 15 addr [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a{3:14625} state up:standby-replay seq 1 addr [v2:172.21.15.49:6840/935071179,v1:172.21.15.49:6841/935071179] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.197 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.l{4:24388} state up:active seq 16 addr [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.198 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.g{4:14613} state up:standby-replay seq 1 addr [v2:172.21.15.49:6838/906046850,v1:172.21.15.49:6839/906046850] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:49.198 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2024-03-26T18:01:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:49.198 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.49:6834/3996463514,v1:172.21.15.49:6835/3996463514] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.198 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.49:6836/2438318193,v1:172.21.15.49:6837/2438318193] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T18:01:49.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 -- 172.21.15.49:0/2012266954 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67840a2fb0 msgr2=0x7f67840a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:49.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67840a2fb0 0x7f67840a5470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6780011490 tx=0x7f6780009260 comp rx=0 tx=0).stop 2024-03-26T18:01:49.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 -- 172.21.15.49:0/2012266954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6798102400 msgr2=0x7f6798198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:49.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6798102400 0x7f6798198570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6790009f20 tx=0x7f679000c3a0 comp rx=0 tx=0).stop 2024-03-26T18:01:49.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 -- 172.21.15.49:0/2012266954 shutdown_connections 2024-03-26T18:01:49.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f67840a2fb0 0x7f67840a5470 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:49.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6798103fa0 0x7f679819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:49.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6798102400 0x7f6798198570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:49.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 --2- 172.21.15.49:0/2012266954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6798103600 0x7f6798198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:49.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 -- 172.21.15.49:0/2012266954 >> 172.21.15.49:0/2012266954 conn(0x7f67980fd9c0 msgr2=0x7f67981068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:49.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 -- 172.21.15.49:0/2012266954 shutdown_connections 2024-03-26T18:01:49.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:49.215+0000 7f679d1f8700 1 -- 172.21.15.49:0/2012266954 wait complete. 2024-03-26T18:01:49.200 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-03-26T18:01:49.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:49 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2012266954' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T18:01:49.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:49 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2012266954' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T18:01:49.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:49 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2012266954' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T18:01:49.880 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-26T18:01:50.138 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:50.515 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:50 smithi049 ceph-mon[32331]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T18:01:50.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:50 smithi115 ceph-mon[36174]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T18:01:50.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:50 smithi171 ceph-mon[36033]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T18:01:51.799 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:51 smithi049 ceph-mon[32331]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-26T18:01:51.799 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:51 smithi049 ceph-mon[32331]: Cluster is now healthy 2024-03-26T18:01:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.822+0000 7f21c17b3700 1 -- 172.21.15.49:0/4188949805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21bc102e80 msgr2=0x7f21bc109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.822+0000 7f21c17b3700 1 --2- 172.21.15.49:0/4188949805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21bc102e80 0x7f21bc109730 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f21b400b0d0 tx=0x7f21b400b3e0 comp rx=0 tx=0).stop 2024-03-26T18:01:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.822+0000 7f21c17b3700 1 -- 172.21.15.49:0/4188949805 shutdown_connections 2024-03-26T18:01:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.822+0000 7f21c17b3700 1 --2- 172.21.15.49:0/4188949805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21bc102e80 0x7f21bc109730 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.822+0000 7f21c17b3700 1 --2- 172.21.15.49:0/4188949805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21bc1012e0 0x7f21bc101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.822+0000 7f21c17b3700 1 --2- 172.21.15.49:0/4188949805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21bc1024e0 0x7f21bc102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.822+0000 7f21c17b3700 1 -- 172.21.15.49:0/4188949805 >> 172.21.15.49:0/4188949805 conn(0x7f21bc0fc880 msgr2=0x7f21bc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.822+0000 7f21c17b3700 1 -- 172.21.15.49:0/4188949805 shutdown_connections 2024-03-26T18:01:51.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.822+0000 7f21c17b3700 1 -- 172.21.15.49:0/4188949805 wait complete. 2024-03-26T18:01:51.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.823+0000 7f21c17b3700 1 Processor -- start 2024-03-26T18:01:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.823+0000 7f21c17b3700 1 -- start start 2024-03-26T18:01:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.823+0000 7f21c17b3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21bc1012e0 0x7f21bc10f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.823+0000 7f21c17b3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21bc1024e0 0x7f21bc10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.823+0000 7f21c17b3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21bc102e80 0x7f21bc10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:51.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.823+0000 7f21c17b3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10cea0 con 0x7f21bc1012e0 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.823+0000 7f21c17b3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10cd20 con 0x7f21bc1024e0 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.823+0000 7f21c17b3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10d020 con 0x7f21bc102e80 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21baffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21bc1012e0 0x7f21bc10f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21ba7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21bc1024e0 0x7f21bc10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21ba7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21bc1024e0 0x7f21bc10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48298/0 (socket says 172.21.15.49:48298) 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21ba7fc700 1 -- 172.21.15.49:0/3815239537 learned_addr learned my addr 172.21.15.49:0/3815239537 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21bb7fe700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21bc102e80 0x7f21bc10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21ba7fc700 1 -- 172.21.15.49:0/3815239537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21bc102e80 msgr2=0x7f21bc10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21ba7fc700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21bc102e80 0x7f21bc10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21ba7fc700 1 -- 172.21.15.49:0/3815239537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21bc1012e0 msgr2=0x7f21bc10f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21ba7fc700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21bc1012e0 0x7f21bc10f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21ba7fc700 1 -- 172.21.15.49:0/3815239537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc10e820 con 0x7f21bc1024e0 2024-03-26T18:01:51.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21baffd700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21bc1012e0 0x7f21bc10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21ba7fc700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21bc1024e0 0x7f21bc10da30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21b000d980 tx=0x7f21b000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21a3fff700 1 -- 172.21.15.49:0/3815239537 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0016070 con 0x7f21bc1024e0 2024-03-26T18:01:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21c17b3700 1 -- 172.21.15.49:0/3815239537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21bc1a2700 con 0x7f21bc1024e0 2024-03-26T18:01:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21a3fff700 1 -- 172.21.15.49:0/3815239537 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21b0004d10 con 0x7f21bc1024e0 2024-03-26T18:01:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21a3fff700 1 -- 172.21.15.49:0/3815239537 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0005670 con 0x7f21bc1024e0 2024-03-26T18:01:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.824+0000 7f21c17b3700 1 -- 172.21.15.49:0/3815239537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc1a2bf0 con 0x7f21bc1024e0 2024-03-26T18:01:51.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.825+0000 7f21c17b3700 1 -- 172.21.15.49:0/3815239537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21bc069070 con 0x7f21bc1024e0 2024-03-26T18:01:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.827+0000 7f21a3fff700 1 -- 172.21.15.49:0/3815239537 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f21b0004750 con 0x7f21bc1024e0 2024-03-26T18:01:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.827+0000 7f21a3fff700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f21a40a2fb0 0x7f21a40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.827+0000 7f21baffd700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f21a40a2fb0 0x7f21a40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.827+0000 7f21a3fff700 1 -- 172.21.15.49:0/3815239537 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f21b00cfa90 con 0x7f21bc1024e0 2024-03-26T18:01:51.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.828+0000 7f21baffd700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f21a40a2fb0 0x7f21a40a5470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f21bc102340 tx=0x7f21ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:51.835+0000 7f21a3fff700 1 -- 172.21.15.49:0/3815239537 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f21b001f020 con 0x7f21bc1024e0 2024-03-26T18:01:51.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:51 smithi171 ceph-mon[36033]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-26T18:01:51.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:51 smithi171 ceph-mon[36033]: Cluster is now healthy 2024-03-26T18:01:51.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:51 smithi115 ceph-mon[36174]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-26T18:01:51.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:51 smithi115 ceph-mon[36174]: Cluster is now healthy 2024-03-26T18:01:52.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.041+0000 7f21c17b3700 1 -- 172.21.15.49:0/3815239537 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f21bc1a3010 con 0x7f21bc1024e0 2024-03-26T18:01:52.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.042+0000 7f21a3fff700 1 -- 172.21.15.49:0/3815239537 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5632 (secure 0 0 0) 0x7f21b009c4e0 con 0x7f21bc1024e0 2024-03-26T18:01:52.024 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 94 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-03-26T17:51:42.367519+0000 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-03-26T18:01:45.470535+0000 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:crush_version 27 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:full_ratio 0.95 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:backfillfull_ratio 0.9 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:nearfull_ratio 0.85 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:require_min_compat_client luminous 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:min_compat_client luminous 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:require_osd_release squid 2024-03-26T18:01:52.025 INFO:teuthology.orchestra.run.smithi049.stdout:stretch_mode_enabled false 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:max_osd 12 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6802/1257601781,v1:172.21.15.49:6803/1257601781] [v2:172.21.15.49:6804/1257601781,v1:172.21.15.49:6805/1257601781] exists,up 22af3c21-ba61-4714-bf75-a3d2761bed59 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/1211178238,v1:172.21.15.115:6801/1211178238] [v2:172.21.15.115:6802/1211178238,v1:172.21.15.115:6803/1211178238] exists,up 68c19f4c-7c26-4834-9bcb-d50b17b767e6 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/2543547141,v1:172.21.15.171:6801/2543547141] [v2:172.21.15.171:6802/2543547141,v1:172.21.15.171:6803/2543547141] exists,up 01428517-2a3a-43fa-88ce-fb0b460d78eb 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6810/3710621844,v1:172.21.15.49:6811/3710621844] [v2:172.21.15.49:6812/3710621844,v1:172.21.15.49:6813/3710621844] exists,up c1b332db-6ccf-45ea-ae70-bd6471ce4e5e 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 up in weight 1 up_from 34 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/1015737283,v1:172.21.15.115:6809/1015737283] [v2:172.21.15.115:6810/1015737283,v1:172.21.15.115:6811/1015737283] exists,up 09354324-9abe-4d61-975b-7c82c3f57a2b 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 up in weight 1 up_from 39 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/2358624537,v1:172.21.15.171:6809/2358624537] [v2:172.21.15.171:6810/2358624537,v1:172.21.15.171:6811/2358624537] exists,up 89af3500-86a4-4dcc-a44d-35ba9af96293 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6818/2242936117,v1:172.21.15.49:6819/2242936117] [v2:172.21.15.49:6820/2242936117,v1:172.21.15.49:6821/2242936117] exists,up 93c76579-30ec-4c2d-a96a-7f939a9cfd3c 2024-03-26T18:01:52.026 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 up in weight 1 up_from 52 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/32789491,v1:172.21.15.115:6817/32789491] [v2:172.21.15.115:6818/32789491,v1:172.21.15.115:6819/32789491] exists,up 7f64173b-87e4-4f35-a827-287345059695 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:osd.8 up in weight 1 up_from 60 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/4095981725,v1:172.21.15.171:6817/4095981725] [v2:172.21.15.171:6818/4095981725,v1:172.21.15.171:6819/4095981725] exists,up a1c2ba0d-b239-4b21-9ecf-ab1a4d948977 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:osd.9 up in weight 1 up_from 68 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6826/1156582500,v1:172.21.15.49:6827/1156582500] [v2:172.21.15.49:6828/1156582500,v1:172.21.15.49:6829/1156582500] exists,up e0c8e333-fbec-4186-90aa-a713043370e3 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:osd.10 up in weight 1 up_from 73 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/2513437602,v1:172.21.15.115:6825/2513437602] [v2:172.21.15.115:6826/2513437602,v1:172.21.15.115:6827/2513437602] exists,up d4828303-f823-4e3c-8298-33bba2eda98d 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/4157105640,v1:172.21.15.171:6825/4157105640] [v2:172.21.15.171:6826/4157105640,v1:172.21.15.171:6827/4157105640] exists,up 8f67590c-f8b2-4ffb-84e5-825d76a17804 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.4 [2,11] 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.6 [8,11] 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.e [4,6,5,9] 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.10 [5,9] 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.1d [4,7] 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.23 [4,8] 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.2d [3,9] 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 4.e [5,8] 2024-03-26T18:01:52.027 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 4.10 [9,10] 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 4.14 [9,8] 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 4.1c [9,8] 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/217046874 expires 2024-03-27T17:52:42.621576+0000 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/442612048 expires 2024-03-27T17:52:42.621576+0000 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/1502160114 expires 2024-03-27T17:52:42.621576+0000 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6801/4249557516 expires 2024-03-27T17:52:42.621576+0000 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/2824939577 expires 2024-03-27T17:52:15.448636+0000 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6800/48778628 expires 2024-03-27T17:52:15.448636+0000 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/1915951880 expires 2024-03-27T17:52:15.448636+0000 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6801/48778628 expires 2024-03-27T17:52:15.448636+0000 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/1560035061 expires 2024-03-27T17:52:15.448636+0000 2024-03-26T18:01:52.028 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6800/4249557516 expires 2024-03-27T17:52:42.621576+0000 2024-03-26T18:01:52.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.045+0000 7f21c17b3700 1 -- 172.21.15.49:0/3815239537 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f21a40a2fb0 msgr2=0x7f21a40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:52.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.045+0000 7f21c17b3700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f21a40a2fb0 0x7f21a40a5470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f21bc102340 tx=0x7f21ac009260 comp rx=0 tx=0).stop 2024-03-26T18:01:52.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.045+0000 7f21c17b3700 1 -- 172.21.15.49:0/3815239537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21bc1024e0 msgr2=0x7f21bc10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:52.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.045+0000 7f21c17b3700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21bc1024e0 0x7f21bc10da30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21b000d980 tx=0x7f21b000dc90 comp rx=0 tx=0).stop 2024-03-26T18:01:52.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.045+0000 7f21c17b3700 1 -- 172.21.15.49:0/3815239537 shutdown_connections 2024-03-26T18:01:52.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.045+0000 7f21c17b3700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f21a40a2fb0 0x7f21a40a5470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:52.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.045+0000 7f21c17b3700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21bc102e80 0x7f21bc10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:52.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.045+0000 7f21c17b3700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21bc1012e0 0x7f21bc10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:52.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.045+0000 7f21c17b3700 1 --2- 172.21.15.49:0/3815239537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21bc1024e0 0x7f21bc10da30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:52.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.045+0000 7f21c17b3700 1 -- 172.21.15.49:0/3815239537 >> 172.21.15.49:0/3815239537 conn(0x7f21bc0fc880 msgr2=0x7f21bc0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:52.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.046+0000 7f21c17b3700 1 -- 172.21.15.49:0/3815239537 shutdown_connections 2024-03-26T18:01:52.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.046+0000 7f21c17b3700 1 -- 172.21.15.49:0/3815239537 wait complete. 2024-03-26T18:01:52.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:52 smithi049 ceph-mon[32331]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-26T18:01:52.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:52 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3815239537' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T18:01:52.817 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-26T18:01:52.830 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:01:52.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:52 smithi171 ceph-mon[36033]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-26T18:01:52.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:52 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3815239537' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T18:01:52.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.963+0000 7f7928953640 1 -- 172.21.15.49:0/3076849273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79241034a0 msgr2=0x7f7924109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:52.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.963+0000 7f7928953640 1 --2- 172.21.15.49:0/3076849273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79241034a0 0x7f7924109d30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f791c009ca0 tx=0x7f791c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:52.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.963+0000 7f7928953640 1 -- 172.21.15.49:0/3076849273 shutdown_connections 2024-03-26T18:01:52.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.963+0000 7f7928953640 1 --2- 172.21.15.49:0/3076849273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79241034a0 0x7f7924109d30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:52.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.963+0000 7f7928953640 1 --2- 172.21.15.49:0/3076849273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7924102ae0 0x7f7924102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:52.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.963+0000 7f7928953640 1 --2- 172.21.15.49:0/3076849273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f79241018f0 0x7f7924101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:52.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.963+0000 7f7928953640 1 -- 172.21.15.49:0/3076849273 >> 172.21.15.49:0/3076849273 conn(0x7f79240fcea0 msgr2=0x7f79240ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:52.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.963+0000 7f7928953640 1 -- 172.21.15.49:0/3076849273 shutdown_connections 2024-03-26T18:01:52.946 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.963+0000 7f7928953640 1 -- 172.21.15.49:0/3076849273 wait complete. 2024-03-26T18:01:52.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7928953640 1 Processor -- start 2024-03-26T18:01:52.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7928953640 1 -- start start 2024-03-26T18:01:52.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7928953640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79241018f0 0x7f7924197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:52.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7928953640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7924102ae0 0x7f79241980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:52.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7928953640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79241034a0 0x7f792419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:52.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7928953640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f792410d550 con 0x7f79241018f0 2024-03-26T18:01:52.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7928953640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f792410d3d0 con 0x7f79241034a0 2024-03-26T18:01:52.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7928953640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f792410d6d0 con 0x7f7924102ae0 2024-03-26T18:01:52.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f792259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79241018f0 0x7f7924197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f792259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79241018f0 0x7f7924197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33052/0 (socket says 172.21.15.49:33052) 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f792259c640 1 -- 172.21.15.49:0/1486583196 learned_addr learned my addr 172.21.15.49:0/1486583196 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7922d9d640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79241034a0 0x7f792419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7921d9b640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7924102ae0 0x7f79241980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f792259c640 1 -- 172.21.15.49:0/1486583196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7924102ae0 msgr2=0x7f79241980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f792259c640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7924102ae0 0x7f79241980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f792259c640 1 -- 172.21.15.49:0/1486583196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79241034a0 msgr2=0x7f792419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f792259c640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79241034a0 0x7f792419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f792259c640 1 -- 172.21.15.49:0/1486583196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792419f7b0 con 0x7f79241018f0 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.964+0000 7f7922d9d640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79241034a0 0x7f792419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:52.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.965+0000 7f792259c640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79241018f0 0x7f7924197b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f791400e9c0 tx=0x7f791400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:52.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.965+0000 7f791b7fe640 1 -- 172.21.15.49:0/1486583196 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791400cd80 con 0x7f79241018f0 2024-03-26T18:01:52.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.965+0000 7f791b7fe640 1 -- 172.21.15.49:0/1486583196 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79140035f0 con 0x7f79241018f0 2024-03-26T18:01:52.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.965+0000 7f7928953640 1 -- 172.21.15.49:0/1486583196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792419faa0 con 0x7f79241018f0 2024-03-26T18:01:52.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.965+0000 7f791b7fe640 1 -- 172.21.15.49:0/1486583196 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914010690 con 0x7f79241018f0 2024-03-26T18:01:52.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.965+0000 7f7928953640 1 -- 172.21.15.49:0/1486583196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792419fff0 con 0x7f79241018f0 2024-03-26T18:01:52.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.966+0000 7f7928953640 1 -- 172.21.15.49:0/1486583196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7924108c10 con 0x7f79241018f0 2024-03-26T18:01:52.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.967+0000 7f791b7fe640 1 -- 172.21.15.49:0/1486583196 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f79140037a0 con 0x7f79241018f0 2024-03-26T18:01:52.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.967+0000 7f791b7fe640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f79100a2f20 0x7f79100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:52.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.968+0000 7f791b7fe640 1 -- 172.21.15.49:0/1486583196 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f79140cfb40 con 0x7f79241018f0 2024-03-26T18:01:52.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.968+0000 7f7921d9b640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f79100a2f20 0x7f79100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:52.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.968+0000 7f7921d9b640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f79100a2f20 0x7f79100a53e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f79241990a0 tx=0x7f790c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:52.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:52.972+0000 7f791b7fe640 1 -- 172.21.15.49:0/1486583196 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f791409c5f0 con 0x7f79241018f0 2024-03-26T18:01:52.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:52 smithi115 ceph-mon[36174]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-26T18:01:52.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:52 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3815239537' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T18:01:53.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.157+0000 7f7928953640 1 -- 172.21.15.49:0/1486583196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79240624c0 con 0x7f79241018f0 2024-03-26T18:01:53.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.159+0000 7f791b7fe640 1 -- 172.21.15.49:0/1486583196 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24672 (secure 0 0 0) 0x7f79140bc0e0 con 0x7f79241018f0 2024-03-26T18:01:53.141 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:53.142 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":94,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T18:01:45.470535+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 -- 172.21.15.49:0/1486583196 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f79100a2f20 msgr2=0x7f79100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f79100a2f20 0x7f79100a53e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f79241990a0 tx=0x7f790c005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 -- 172.21.15.49:0/1486583196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79241018f0 msgr2=0x7f7924197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79241018f0 0x7f7924197b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f791400e9c0 tx=0x7f791400ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 -- 172.21.15.49:0/1486583196 shutdown_connections 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f79100a2f20 0x7f79100a53e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7924102ae0 0x7f79241980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79241034a0 0x7f792419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 --2- 172.21.15.49:0/1486583196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79241018f0 0x7f7924197b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 -- 172.21.15.49:0/1486583196 >> 172.21.15.49:0/1486583196 conn(0x7f79240fcea0 msgr2=0x7f7924105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 -- 172.21.15.49:0/1486583196 shutdown_connections 2024-03-26T18:01:53.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.161+0000 7f7928953640 1 -- 172.21.15.49:0/1486583196 wait complete. 2024-03-26T18:01:53.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:59:45.382542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:59:46.221139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T17:59:49.890383+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T18:01:53.156 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:01:53.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.285+0000 7fd00b2f8640 1 -- 172.21.15.49:0/4200518077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0041007e0 msgr2=0x7fd004100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.285+0000 7fd00b2f8640 1 --2- 172.21.15.49:0/4200518077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0041007e0 0x7fd004100c00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd000009900 tx=0x7fd00002bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:53.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.285+0000 7fd00b2f8640 1 -- 172.21.15.49:0/4200518077 shutdown_connections 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.285+0000 7fd00b2f8640 1 --2- 172.21.15.49:0/4200518077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0041019d0 0x7fd004101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.285+0000 7fd00b2f8640 1 --2- 172.21.15.49:0/4200518077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd004102390 0x7fd004108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.285+0000 7fd00b2f8640 1 --2- 172.21.15.49:0/4200518077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0041007e0 0x7fd004100c00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.285+0000 7fd00b2f8640 1 -- 172.21.15.49:0/4200518077 >> 172.21.15.49:0/4200518077 conn(0x7fd0040fbd70 msgr2=0x7fd0040fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.285+0000 7fd00b2f8640 1 -- 172.21.15.49:0/4200518077 shutdown_connections 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.285+0000 7fd00b2f8640 1 -- 172.21.15.49:0/4200518077 wait complete. 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd00b2f8640 1 Processor -- start 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd00b2f8640 1 -- start start 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd00b2f8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0041007e0 0x7fd004197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd00b2f8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0041019d0 0x7fd004198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd00b2f8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd004102390 0x7fd00419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd00b2f8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd00410c430 con 0x7fd0041019d0 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd00b2f8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd00410c2b0 con 0x7fd0041007e0 2024-03-26T18:01:53.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd00b2f8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd00410c5b0 con 0x7fd004102390 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd008893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0041019d0 0x7fd004198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd008893640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0041019d0 0x7fd004198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33064/0 (socket says 172.21.15.49:33064) 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd008893640 1 -- 172.21.15.49:0/3486582639 learned_addr learned my addr 172.21.15.49:0/3486582639 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd009094640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0041007e0 0x7fd004197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd009895640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd004102390 0x7fd00419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd008893640 1 -- 172.21.15.49:0/3486582639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd004102390 msgr2=0x7fd00419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd008893640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd004102390 0x7fd00419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd008893640 1 -- 172.21.15.49:0/3486582639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0041007e0 msgr2=0x7fd004197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd008893640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0041007e0 0x7fd004197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd008893640 1 -- 172.21.15.49:0/3486582639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00419f890 con 0x7fd0041019d0 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.286+0000 7fd008893640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0041019d0 0x7fd004198210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcff400ea60 tx=0x7fcff400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.287+0000 7fcffa7fc640 1 -- 172.21.15.49:0/3486582639 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff400cde0 con 0x7fd0041019d0 2024-03-26T18:01:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.287+0000 7fd00b2f8640 1 -- 172.21.15.49:0/3486582639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00419fb80 con 0x7fd0041019d0 2024-03-26T18:01:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.287+0000 7fcffa7fc640 1 -- 172.21.15.49:0/3486582639 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff40035f0 con 0x7fd0041019d0 2024-03-26T18:01:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.287+0000 7fcffa7fc640 1 -- 172.21.15.49:0/3486582639 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4010670 con 0x7fd0041019d0 2024-03-26T18:01:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.287+0000 7fd00b2f8640 1 -- 172.21.15.49:0/3486582639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0041a00f0 con 0x7fd0041019d0 2024-03-26T18:01:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.288+0000 7fd00b2f8640 1 -- 172.21.15.49:0/3486582639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0041a02c0 con 0x7fd0041019d0 2024-03-26T18:01:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.290+0000 7fcffa7fc640 1 -- 172.21.15.49:0/3486582639 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fcff40037a0 con 0x7fd0041019d0 2024-03-26T18:01:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.290+0000 7fcffa7fc640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fcff00a2f20 0x7fcff00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.290+0000 7fd009094640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fcff00a2f20 0x7fcff00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.290+0000 7fcffa7fc640 1 -- 172.21.15.49:0/3486582639 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7fcff40cfb70 con 0x7fd0041019d0 2024-03-26T18:01:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.291+0000 7fd009094640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fcff00a2f20 0x7fcff00a53e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd000000c00 tx=0x7fd000005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.294+0000 7fcffa7fc640 1 -- 172.21.15.49:0/3486582639 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcff409c620 con 0x7fd0041019d0 2024-03-26T18:01:53.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.478+0000 7fd00b2f8640 1 -- 172.21.15.49:0/3486582639 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0041a0490 con 0x7fd0041019d0 2024-03-26T18:01:53.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.479+0000 7fcffa7fc640 1 -- 172.21.15.49:0/3486582639 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fcff409bf20 con 0x7fd0041019d0 2024-03-26T18:01:53.461 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T18:01:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.480+0000 7fd00b2f8640 1 -- 172.21.15.49:0/3486582639 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fcff00a2f20 msgr2=0x7fcff00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.480+0000 7fd00b2f8640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fcff00a2f20 0x7fcff00a53e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd000000c00 tx=0x7fd000005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.481+0000 7fd00b2f8640 1 -- 172.21.15.49:0/3486582639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0041019d0 msgr2=0x7fd004198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.481+0000 7fd00b2f8640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0041019d0 0x7fd004198210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcff400ea60 tx=0x7fcff400ef20 comp rx=0 tx=0).stop 2024-03-26T18:01:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.481+0000 7fd00b2f8640 1 -- 172.21.15.49:0/3486582639 shutdown_connections 2024-03-26T18:01:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.481+0000 7fd00b2f8640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fcff00a2f20 0x7fcff00a53e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.481+0000 7fd00b2f8640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd004102390 0x7fd00419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.481+0000 7fd00b2f8640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0041019d0 0x7fd004198210 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.481+0000 7fd00b2f8640 1 --2- 172.21.15.49:0/3486582639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0041007e0 0x7fd004197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.481+0000 7fd00b2f8640 1 -- 172.21.15.49:0/3486582639 >> 172.21.15.49:0/3486582639 conn(0x7fd0040fbd70 msgr2=0x7fd0040fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.481+0000 7fd00b2f8640 1 -- 172.21.15.49:0/3486582639 shutdown_connections 2024-03-26T18:01:53.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.481+0000 7fd00b2f8640 1 -- 172.21.15.49:0/3486582639 wait complete. 2024-03-26T18:01:53.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:01:53.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.604+0000 7fbd2c759640 1 -- 172.21.15.49:0/1063607356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd24102340 msgr2=0x7fbd24104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.604+0000 7fbd2c759640 1 --2- 172.21.15.49:0/1063607356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd24102340 0x7fbd24104730 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009900 tx=0x7fbd2002bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:53.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.604+0000 7fbd2c759640 1 -- 172.21.15.49:0/1063607356 shutdown_connections 2024-03-26T18:01:53.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.604+0000 7fbd2c759640 1 --2- 172.21.15.49:0/1063607356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd241075a0 0x7fbd241099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.604+0000 7fbd2c759640 1 --2- 172.21.15.49:0/1063607356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd24104c70 0x7fbd24107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.604+0000 7fbd2c759640 1 --2- 172.21.15.49:0/1063607356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd24102340 0x7fbd24104730 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.604+0000 7fbd2c759640 1 -- 172.21.15.49:0/1063607356 >> 172.21.15.49:0/1063607356 conn(0x7fbd240fbd50 msgr2=0x7fbd240fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.604+0000 7fbd2c759640 1 -- 172.21.15.49:0/1063607356 shutdown_connections 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.604+0000 7fbd2c759640 1 -- 172.21.15.49:0/1063607356 wait complete. 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd2c759640 1 Processor -- start 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd2c759640 1 -- start start 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd2c759640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd24102340 0x7fbd24197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd2c759640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd24104c70 0x7fbd24198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd2c759640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd241075a0 0x7fbd2419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd2c759640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2410d560 con 0x7fbd24104c70 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd2c759640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2410d3e0 con 0x7fbd241075a0 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd2c759640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2410d6e0 con 0x7fbd24102340 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd29cf4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd24104c70 0x7fbd24198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd29cf4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd24104c70 0x7fbd24198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33072/0 (socket says 172.21.15.49:33072) 2024-03-26T18:01:53.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd29cf4640 1 -- 172.21.15.49:0/3024036574 learned_addr learned my addr 172.21.15.49:0/3024036574 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd2acf6640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd241075a0 0x7fbd2419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd2a4f5640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd24102340 0x7fbd24197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd29cf4640 1 -- 172.21.15.49:0/3024036574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd24102340 msgr2=0x7fbd24197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd29cf4640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd24102340 0x7fbd24197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd29cf4640 1 -- 172.21.15.49:0/3024036574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd241075a0 msgr2=0x7fbd2419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd29cf4640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd241075a0 0x7fbd2419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd29cf4640 1 -- 172.21.15.49:0/3024036574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2419f860 con 0x7fbd24104c70 2024-03-26T18:01:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.605+0000 7fbd29cf4640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd24104c70 0x7fbd24198190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400e9c0 tx=0x7fbd1400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.606+0000 7fbd1b7fe640 1 -- 172.21.15.49:0/3024036574 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400cd80 con 0x7fbd24104c70 2024-03-26T18:01:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.606+0000 7fbd1b7fe640 1 -- 172.21.15.49:0/3024036574 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd14005500 con 0x7fbd24104c70 2024-03-26T18:01:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.606+0000 7fbd2c759640 1 -- 172.21.15.49:0/3024036574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2419fb50 con 0x7fbd24104c70 2024-03-26T18:01:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.606+0000 7fbd1b7fe640 1 -- 172.21.15.49:0/3024036574 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14010690 con 0x7fbd24104c70 2024-03-26T18:01:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.606+0000 7fbd2c759640 1 -- 172.21.15.49:0/3024036574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd241a00a0 con 0x7fbd24104c70 2024-03-26T18:01:53.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.607+0000 7fbd2c759640 1 -- 172.21.15.49:0/3024036574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd240fd840 con 0x7fbd24104c70 2024-03-26T18:01:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.608+0000 7fbd1b7fe640 1 -- 172.21.15.49:0/3024036574 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fbd140046c0 con 0x7fbd24104c70 2024-03-26T18:01:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.608+0000 7fbd1b7fe640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbd100a2f20 0x7fbd100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.608+0000 7fbd2a4f5640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbd100a2f20 0x7fbd100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.608+0000 7fbd1b7fe640 1 -- 172.21.15.49:0/3024036574 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7fbd140cfd90 con 0x7fbd24104c70 2024-03-26T18:01:53.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.609+0000 7fbd2a4f5640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbd100a2f20 0x7fbd100a53e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbd24199120 tx=0x7fbd20005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.612+0000 7fbd1b7fe640 1 -- 172.21.15.49:0/3024036574 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbd14053740 con 0x7fbd24104c70 2024-03-26T18:01:53.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:53 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1486583196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:53.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:53 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3486582639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:53.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.794+0000 7fbd2c759640 1 -- 172.21.15.49:0/3024036574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd2402d490 con 0x7fbd24104c70 2024-03-26T18:01:53.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.794+0000 7fbd1b7fe640 1 -- 172.21.15.49:0/3024036574 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fbd1409c840 con 0x7fbd24104c70 2024-03-26T18:01:53.777 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:01:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 -- 172.21.15.49:0/3024036574 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbd100a2f20 msgr2=0x7fbd100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbd100a2f20 0x7fbd100a53e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbd24199120 tx=0x7fbd20005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 -- 172.21.15.49:0/3024036574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd24104c70 msgr2=0x7fbd24198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd24104c70 0x7fbd24198190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400e9c0 tx=0x7fbd1400ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 -- 172.21.15.49:0/3024036574 shutdown_connections 2024-03-26T18:01:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fbd100a2f20 0x7fbd100a53e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd241075a0 0x7fbd2419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd24104c70 0x7fbd24198190 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 --2- 172.21.15.49:0/3024036574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd24102340 0x7fbd24197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 -- 172.21.15.49:0/3024036574 >> 172.21.15.49:0/3024036574 conn(0x7fbd240fbd50 msgr2=0x7fbd24102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 -- 172.21.15.49:0/3024036574 shutdown_connections 2024-03-26T18:01:53.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.797+0000 7fbd2c759640 1 -- 172.21.15.49:0/3024036574 wait complete. 2024-03-26T18:01:53.790 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:01:53.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:53 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1486583196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:53.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:53 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3486582639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:53.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.919+0000 7f9c9113b640 1 -- 172.21.15.49:0/4163427967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c8c1018e0 msgr2=0x7f9c8c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.919+0000 7f9c9113b640 1 --2- 172.21.15.49:0/4163427967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c8c1018e0 0x7f9c8c101d00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009900 tx=0x7f9c7402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:53.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.920+0000 7f9c9113b640 1 -- 172.21.15.49:0/4163427967 shutdown_connections 2024-03-26T18:01:53.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.920+0000 7f9c9113b640 1 --2- 172.21.15.49:0/4163427967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c8c103490 0x7f9c8c109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.920+0000 7f9c9113b640 1 --2- 172.21.15.49:0/4163427967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c102ad0 0x7f9c8c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.920+0000 7f9c9113b640 1 --2- 172.21.15.49:0/4163427967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c8c1018e0 0x7f9c8c101d00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.920+0000 7f9c9113b640 1 -- 172.21.15.49:0/4163427967 >> 172.21.15.49:0/4163427967 conn(0x7f9c8c0fce50 msgr2=0x7f9c8c0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.920+0000 7f9c9113b640 1 -- 172.21.15.49:0/4163427967 shutdown_connections 2024-03-26T18:01:53.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.920+0000 7f9c9113b640 1 -- 172.21.15.49:0/4163427967 wait complete. 2024-03-26T18:01:53.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.920+0000 7f9c9113b640 1 Processor -- start 2024-03-26T18:01:53.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.920+0000 7f9c9113b640 1 -- start start 2024-03-26T18:01:53.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c9113b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c1018e0 0x7f9c8c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c9113b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c8c102ad0 0x7f9c8c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c9113b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c8c103490 0x7f9c8c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c9113b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c10d560 con 0x7f9c8c1018e0 2024-03-26T18:01:53.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c9113b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c10d3e0 con 0x7f9c8c102ad0 2024-03-26T18:01:53.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c9113b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c10d6e0 con 0x7f9c8c103490 2024-03-26T18:01:53.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c1018e0 0x7f9c8c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c1018e0 0x7f9c8c197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33088/0 (socket says 172.21.15.49:33088) 2024-03-26T18:01:53.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8ad9d640 1 -- 172.21.15.49:0/4143862887 learned_addr learned my addr 172.21.15.49:0/4143862887 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:53.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8a59c640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c8c102ad0 0x7f9c8c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8b59e640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c8c103490 0x7f9c8c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8a59c640 1 -- 172.21.15.49:0/4143862887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c8c103490 msgr2=0x7f9c8c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8a59c640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c8c103490 0x7f9c8c19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8a59c640 1 -- 172.21.15.49:0/4143862887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c1018e0 msgr2=0x7f9c8c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8a59c640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c1018e0 0x7f9c8c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8a59c640 1 -- 172.21.15.49:0/4143862887 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8c19f800 con 0x7f9c8c102ad0 2024-03-26T18:01:53.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8ad9d640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c1018e0 0x7f9c8c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:53.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c8a59c640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c8c102ad0 0x7f9c8c198120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00e9f0 tx=0x7f9c7c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.921+0000 7f9c83fff640 1 -- 172.21.15.49:0/4143862887 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00ccc0 con 0x7f9c8c102ad0 2024-03-26T18:01:53.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.922+0000 7f9c83fff640 1 -- 172.21.15.49:0/4143862887 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7c010910 con 0x7f9c8c102ad0 2024-03-26T18:01:53.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.922+0000 7f9c9113b640 1 -- 172.21.15.49:0/4143862887 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8c19faf0 con 0x7f9c8c102ad0 2024-03-26T18:01:53.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.922+0000 7f9c83fff640 1 -- 172.21.15.49:0/4143862887 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c018960 con 0x7f9c8c102ad0 2024-03-26T18:01:53.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.922+0000 7f9c9113b640 1 -- 172.21.15.49:0/4143862887 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8c1a0040 con 0x7f9c8c102ad0 2024-03-26T18:01:53.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.923+0000 7f9c9113b640 1 -- 172.21.15.49:0/4143862887 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8c108c00 con 0x7f9c8c102ad0 2024-03-26T18:01:53.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.924+0000 7f9c83fff640 1 -- 172.21.15.49:0/4143862887 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f9c7c010ac0 con 0x7f9c8c102ad0 2024-03-26T18:01:53.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.924+0000 7f9c83fff640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9c780a2f20 0x7f9c780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.924+0000 7f9c8ad9d640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9c780a2f20 0x7f9c780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.924+0000 7f9c83fff640 1 -- 172.21.15.49:0/4143862887 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f9c7c0cfe10 con 0x7f9c8c102ad0 2024-03-26T18:01:53.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.925+0000 7f9c8ad9d640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9c780a2f20 0x7f9c780a53e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c7402c8e0 tx=0x7f9c74005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:53.928+0000 7f9c83fff640 1 -- 172.21.15.49:0/4143862887 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9c7c09c270 con 0x7f9c8c102ad0 2024-03-26T18:01:53.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:53 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1486583196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:53.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:53 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3486582639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:54.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.109+0000 7f9c9113b640 1 -- 172.21.15.49:0/4143862887 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9c8c02d440 con 0x7f9c8c102ad0 2024-03-26T18:01:54.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.110+0000 7f9c83fff640 1 -- 172.21.15.49:0/4143862887 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f9c7c0a0310 con 0x7f9c8c102ad0 2024-03-26T18:01:54.092 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-26T18:01:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 -- 172.21.15.49:0/4143862887 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9c780a2f20 msgr2=0x7f9c780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9c780a2f20 0x7f9c780a53e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c7402c8e0 tx=0x7f9c74005b50 comp rx=0 tx=0).stop 2024-03-26T18:01:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 -- 172.21.15.49:0/4143862887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c8c102ad0 msgr2=0x7f9c8c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c8c102ad0 0x7f9c8c198120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00e9f0 tx=0x7f9c7c00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 -- 172.21.15.49:0/4143862887 shutdown_connections 2024-03-26T18:01:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f9c780a2f20 0x7f9c780a53e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c8c103490 0x7f9c8c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c8c102ad0 0x7f9c8c198120 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 --2- 172.21.15.49:0/4143862887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c8c1018e0 0x7f9c8c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 -- 172.21.15.49:0/4143862887 >> 172.21.15.49:0/4143862887 conn(0x7f9c8c0fce50 msgr2=0x7f9c8c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 -- 172.21.15.49:0/4143862887 shutdown_connections 2024-03-26T18:01:54.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.112+0000 7f9c9113b640 1 -- 172.21.15.49:0/4143862887 wait complete. 2024-03-26T18:01:54.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T18:01:54.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.233+0000 7f40710e8640 1 -- 172.21.15.49:0/4155389352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1019d0 msgr2=0x7f406c101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.233+0000 7f40710e8640 1 --2- 172.21.15.49:0/4155389352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1019d0 0x7f406c101e50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4060009900 tx=0x7f406002bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:54.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.233+0000 7f40710e8640 1 -- 172.21.15.49:0/4155389352 shutdown_connections 2024-03-26T18:01:54.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.233+0000 7f40710e8640 1 --2- 172.21.15.49:0/4155389352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406c102390 0x7f406c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.233+0000 7f40710e8640 1 --2- 172.21.15.49:0/4155389352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1019d0 0x7f406c101e50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.233+0000 7f40710e8640 1 --2- 172.21.15.49:0/4155389352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f406c1007e0 0x7f406c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.233+0000 7f40710e8640 1 -- 172.21.15.49:0/4155389352 >> 172.21.15.49:0/4155389352 conn(0x7f406c0fbd70 msgr2=0x7f406c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:54.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.234+0000 7f40710e8640 1 -- 172.21.15.49:0/4155389352 shutdown_connections 2024-03-26T18:01:54.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.234+0000 7f40710e8640 1 -- 172.21.15.49:0/4155389352 wait complete. 2024-03-26T18:01:54.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.234+0000 7f40710e8640 1 Processor -- start 2024-03-26T18:01:54.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.234+0000 7f40710e8640 1 -- start start 2024-03-26T18:01:54.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f40710e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1007e0 0x7f406c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f40710e8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f406c1019d0 0x7f406c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f40710e8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406c102390 0x7f406c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f40710e8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f406c10c460 con 0x7f406c1007e0 2024-03-26T18:01:54.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f40710e8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f406c10c2e0 con 0x7f406c1019d0 2024-03-26T18:01:54.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f40710e8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f406c10c5e0 con 0x7f406c102390 2024-03-26T18:01:54.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1007e0 0x7f406c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1007e0 0x7f406c199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33108/0 (socket says 172.21.15.49:33108) 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406ad9d640 1 -- 172.21.15.49:0/159096735 learned_addr learned my addr 172.21.15.49:0/159096735 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406a59c640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f406c1019d0 0x7f406c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406b59e640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406c102390 0x7f406c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406ad9d640 1 -- 172.21.15.49:0/159096735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406c102390 msgr2=0x7f406c1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406ad9d640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406c102390 0x7f406c1a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406ad9d640 1 -- 172.21.15.49:0/159096735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f406c1019d0 msgr2=0x7f406c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406ad9d640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f406c1019d0 0x7f406c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406ad9d640 1 -- 172.21.15.49:0/159096735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c1a1a50 con 0x7f406c1007e0 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f406ad9d640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1007e0 0x7f406c199e30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f405c00e9c0 tx=0x7f405c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f4053fff640 1 -- 172.21.15.49:0/159096735 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c00cd80 con 0x7f406c1007e0 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f4053fff640 1 -- 172.21.15.49:0/159096735 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f405c0035f0 con 0x7f406c1007e0 2024-03-26T18:01:54.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f40710e8640 1 -- 172.21.15.49:0/159096735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c1a1d40 con 0x7f406c1007e0 2024-03-26T18:01:54.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.235+0000 7f4053fff640 1 -- 172.21.15.49:0/159096735 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c010690 con 0x7f406c1007e0 2024-03-26T18:01:54.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.236+0000 7f40710e8640 1 -- 172.21.15.49:0/159096735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c1a21b0 con 0x7f406c1007e0 2024-03-26T18:01:54.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.236+0000 7f40710e8640 1 -- 172.21.15.49:0/159096735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c1a2440 con 0x7f406c1007e0 2024-03-26T18:01:54.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.237+0000 7f4053fff640 1 -- 172.21.15.49:0/159096735 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f405c0037a0 con 0x7f406c1007e0 2024-03-26T18:01:54.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.238+0000 7f4053fff640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f40540a2f20 0x7f40540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.238+0000 7f406a59c640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f40540a2f20 0x7f40540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.238+0000 7f4053fff640 1 -- 172.21.15.49:0/159096735 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f405c0cfb40 con 0x7f406c1007e0 2024-03-26T18:01:54.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.239+0000 7f406a59c640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f40540a2f20 0x7f40540a53e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f406002c7b0 tx=0x7f4060005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:54.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.243+0000 7f4053fff640 1 -- 172.21.15.49:0/159096735 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f405c09c5f0 con 0x7f406c1007e0 2024-03-26T18:01:54.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.427+0000 7f40710e8640 1 -- 172.21.15.49:0/159096735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f406c1a2ce0 con 0x7f406c1007e0 2024-03-26T18:01:54.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.427+0000 7f4053fff640 1 -- 172.21.15.49:0/159096735 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f405c09bef0 con 0x7f406c1007e0 2024-03-26T18:01:54.409 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:01:54.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 -- 172.21.15.49:0/159096735 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f40540a2f20 msgr2=0x7f40540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f40540a2f20 0x7f40540a53e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f406002c7b0 tx=0x7f4060005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 -- 172.21.15.49:0/159096735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1007e0 msgr2=0x7f406c199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1007e0 0x7f406c199e30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f405c00e9c0 tx=0x7f405c00ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 -- 172.21.15.49:0/159096735 shutdown_connections 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f40540a2f20 0x7f40540a53e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f406c102390 0x7f406c1a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f406c1019d0 0x7f406c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 --2- 172.21.15.49:0/159096735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f406c1007e0 0x7f406c199e30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 -- 172.21.15.49:0/159096735 >> 172.21.15.49:0/159096735 conn(0x7f406c0fbd70 msgr2=0x7f406c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 -- 172.21.15.49:0/159096735 shutdown_connections 2024-03-26T18:01:54.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.429+0000 7f40710e8640 1 -- 172.21.15.49:0/159096735 wait complete. 2024-03-26T18:01:54.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:01:54.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.550+0000 7f7d40885640 1 -- 172.21.15.49:0/1849683677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c1007e0 msgr2=0x7f7d3c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.550+0000 7f7d40885640 1 --2- 172.21.15.49:0/1849683677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c1007e0 0x7f7d3c100c00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009900 tx=0x7f7d2c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:54.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.550+0000 7f7d40885640 1 -- 172.21.15.49:0/1849683677 shutdown_connections 2024-03-26T18:01:54.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.550+0000 7f7d40885640 1 --2- 172.21.15.49:0/1849683677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d3c1019d0 0x7f7d3c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.550+0000 7f7d40885640 1 --2- 172.21.15.49:0/1849683677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c102390 0x7f7d3c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.550+0000 7f7d40885640 1 --2- 172.21.15.49:0/1849683677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c1007e0 0x7f7d3c100c00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.550+0000 7f7d40885640 1 -- 172.21.15.49:0/1849683677 >> 172.21.15.49:0/1849683677 conn(0x7f7d3c0fbd50 msgr2=0x7f7d3c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:54.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.550+0000 7f7d40885640 1 -- 172.21.15.49:0/1849683677 shutdown_connections 2024-03-26T18:01:54.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.550+0000 7f7d40885640 1 -- 172.21.15.49:0/1849683677 wait complete. 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d40885640 1 Processor -- start 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d40885640 1 -- start start 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d40885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c1007e0 0x7f7d3c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d40885640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c1019d0 0x7f7d3c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d40885640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d3c102390 0x7f7d3c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d40885640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c10d580 con 0x7f7d3c1007e0 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d40885640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c10d400 con 0x7f7d3c1019d0 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d40885640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c10d700 con 0x7f7d3c102390 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d3a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c1007e0 0x7f7d3c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d3a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c1007e0 0x7f7d3c197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33134/0 (socket says 172.21.15.49:33134) 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d3a59c640 1 -- 172.21.15.49:0/2448140517 learned_addr learned my addr 172.21.15.49:0/2448140517 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d33fff640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c1019d0 0x7f7d3c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d3ad9d640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d3c102390 0x7f7d3c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d3a59c640 1 -- 172.21.15.49:0/2448140517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d3c102390 msgr2=0x7f7d3c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.551+0000 7f7d3a59c640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d3c102390 0x7f7d3c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.552+0000 7f7d3a59c640 1 -- 172.21.15.49:0/2448140517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c1019d0 msgr2=0x7f7d3c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.552+0000 7f7d3a59c640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c1019d0 0x7f7d3c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.552+0000 7f7d3a59c640 1 -- 172.21.15.49:0/2448140517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c19f860 con 0x7f7d3c1007e0 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.552+0000 7f7d3a59c640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c1007e0 0x7f7d3c197c50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c000c00 tx=0x7f7d2c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.552+0000 7f7d337fe640 1 -- 172.21.15.49:0/2448140517 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c041070 con 0x7f7d3c1007e0 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.552+0000 7f7d40885640 1 -- 172.21.15.49:0/2448140517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c19faf0 con 0x7f7d3c1007e0 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.552+0000 7f7d337fe640 1 -- 172.21.15.49:0/2448140517 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c004450 con 0x7f7d3c1007e0 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.552+0000 7f7d337fe640 1 -- 172.21.15.49:0/2448140517 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c034440 con 0x7f7d3c1007e0 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.552+0000 7f7d40885640 1 -- 172.21.15.49:0/2448140517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c19ffe0 con 0x7f7d3c1007e0 2024-03-26T18:01:54.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.553+0000 7f7d40885640 1 -- 172.21.15.49:0/2448140517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3c107b50 con 0x7f7d3c1007e0 2024-03-26T18:01:54.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.554+0000 7f7d337fe640 1 -- 172.21.15.49:0/2448140517 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f7d2c044070 con 0x7f7d3c1007e0 2024-03-26T18:01:54.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.555+0000 7f7d337fe640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7d280a2f20 0x7f7d280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.555+0000 7f7d33fff640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7d280a2f20 0x7f7d280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.555+0000 7f7d337fe640 1 -- 172.21.15.49:0/2448140517 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f7d2c0f1490 con 0x7f7d3c1007e0 2024-03-26T18:01:54.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.555+0000 7f7d33fff640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7d280a2f20 0x7f7d280a53e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009bd0 tx=0x7f7d24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:54.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.558+0000 7f7d337fe640 1 -- 172.21.15.49:0/2448140517 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7d2c0bd8f0 con 0x7f7d3c1007e0 2024-03-26T18:01:54.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:54 smithi049 ceph-mon[32331]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-26T18:01:54.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:54 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3024036574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:54.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:54 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4143862887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:54.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:54 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/159096735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:54.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.780+0000 7f7d40885640 1 -- 172.21.15.49:0/2448140517 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d3c02d490 con 0x7f7d3c1007e0 2024-03-26T18:01:54.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.780+0000 7f7d337fe640 1 -- 172.21.15.49:0/2448140517 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11943 (secure 0 0 0) 0x7f7d2c0c1990 con 0x7f7d3c1007e0 2024-03-26T18:01:54.763 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:54.763 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:59:46.796381+0000","modified":"2024-03-26T18:00:09.450708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24503,"mds_2":24478,"mds_3":24428,"mds_4":24388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24478,24503,24388,24428,24533]},"id":1}]} 2024-03-26T18:01:54.763 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-03-26T18:01:54.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 -- 172.21.15.49:0/2448140517 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7d280a2f20 msgr2=0x7f7d280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7d280a2f20 0x7f7d280a53e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009bd0 tx=0x7f7d24005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:54.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 -- 172.21.15.49:0/2448140517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c1007e0 msgr2=0x7f7d3c197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c1007e0 0x7f7d3c197c50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c000c00 tx=0x7f7d2c02c640 comp rx=0 tx=0).stop 2024-03-26T18:01:54.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 -- 172.21.15.49:0/2448140517 shutdown_connections 2024-03-26T18:01:54.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7d280a2f20 0x7f7d280a53e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d3c102390 0x7f7d3c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c1019d0 0x7f7d3c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 --2- 172.21.15.49:0/2448140517 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d3c1007e0 0x7f7d3c197c50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 -- 172.21.15.49:0/2448140517 >> 172.21.15.49:0/2448140517 conn(0x7f7d3c0fbd50 msgr2=0x7f7d3c104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:54.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 -- 172.21.15.49:0/2448140517 shutdown_connections 2024-03-26T18:01:54.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.783+0000 7f7d40885640 1 -- 172.21.15.49:0/2448140517 wait complete. 2024-03-26T18:01:54.776 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:01:54.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:54 smithi171 ceph-mon[36033]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-26T18:01:54.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:54 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3024036574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:54.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:54 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4143862887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:54.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:54 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/159096735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:54.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.902+0000 7f0ccaec0640 1 -- 172.21.15.49:0/3428947276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc4101a50 msgr2=0x7f0cc4101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.902+0000 7f0ccaec0640 1 --2- 172.21.15.49:0/3428947276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc4101a50 0x7f0cc4101ed0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009860 tx=0x7f0cb402bb70 comp rx=0 tx=0).stop 2024-03-26T18:01:54.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.902+0000 7f0ccaec0640 1 -- 172.21.15.49:0/3428947276 shutdown_connections 2024-03-26T18:01:54.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.902+0000 7f0ccaec0640 1 --2- 172.21.15.49:0/3428947276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4102410 0x7f0cc4108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.902+0000 7f0ccaec0640 1 --2- 172.21.15.49:0/3428947276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc4101a50 0x7f0cc4101ed0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.902+0000 7f0ccaec0640 1 --2- 172.21.15.49:0/3428947276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc4100860 0x7f0cc4100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.902+0000 7f0ccaec0640 1 -- 172.21.15.49:0/3428947276 >> 172.21.15.49:0/3428947276 conn(0x7f0cc40fbdf0 msgr2=0x7f0cc40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:54.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.902+0000 7f0ccaec0640 1 -- 172.21.15.49:0/3428947276 shutdown_connections 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.902+0000 7f0ccaec0640 1 -- 172.21.15.49:0/3428947276 wait complete. 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0ccaec0640 1 Processor -- start 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0ccaec0640 1 -- start start 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0ccaec0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc4100860 0x7f0cc406aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0ccaec0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc4101a50 0x7f0cc419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0ccaec0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4102410 0x7f0cc419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0ccaec0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410c450 con 0x7f0cc4101a50 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0ccaec0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410c2d0 con 0x7f0cc4100860 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0ccaec0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410c5d0 con 0x7f0cc4102410 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0cbbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc4101a50 0x7f0cc419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0cc8c5c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc4100860 0x7f0cc406aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0cbbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc4101a50 0x7f0cc419c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33154/0 (socket says 172.21.15.49:33154) 2024-03-26T18:01:54.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0cbbfff640 1 -- 172.21.15.49:0/4209289106 learned_addr learned my addr 172.21.15.49:0/4209289106 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.903+0000 7f0cc945d640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4102410 0x7f0cc419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0cbbfff640 1 -- 172.21.15.49:0/4209289106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4102410 msgr2=0x7f0cc419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0cbbfff640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4102410 0x7f0cc419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0cbbfff640 1 -- 172.21.15.49:0/4209289106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc4100860 msgr2=0x7f0cc406aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0cbbfff640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc4100860 0x7f0cc406aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0cbbfff640 1 -- 172.21.15.49:0/4209289106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc419f840 con 0x7f0cc4101a50 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0cbbfff640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc4101a50 0x7f0cc419c910 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4005940 tx=0x7f0cb4032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0cb9ffb640 1 -- 172.21.15.49:0/4209289106 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4041020 con 0x7f0cc4101a50 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0cb9ffb640 1 -- 172.21.15.49:0/4209289106 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cb402c9f0 con 0x7f0cc4101a50 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0cb9ffb640 1 -- 172.21.15.49:0/4209289106 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4039040 con 0x7f0cc4101a50 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0ccaec0640 1 -- 172.21.15.49:0/4209289106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc419fa70 con 0x7f0cc4101a50 2024-03-26T18:01:54.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.904+0000 7f0ccaec0640 1 -- 172.21.15.49:0/4209289106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc419ff10 con 0x7f0cc4101a50 2024-03-26T18:01:54.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.905+0000 7f0ccaec0640 1 -- 172.21.15.49:0/4209289106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc4107bd0 con 0x7f0cc4101a50 2024-03-26T18:01:54.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.906+0000 7f0cb9ffb640 1 -- 172.21.15.49:0/4209289106 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f0cb4044020 con 0x7f0cc4101a50 2024-03-26T18:01:54.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.907+0000 7f0cb9ffb640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f0cac0a2f20 0x7f0cac0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:54.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.907+0000 7f0cc8c5c640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f0cac0a2f20 0x7f0cac0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:54.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.907+0000 7f0cb9ffb640 1 -- 172.21.15.49:0/4209289106 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f0cb40f10f0 con 0x7f0cc4101a50 2024-03-26T18:01:54.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.908+0000 7f0cc8c5c640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f0cac0a2f20 0x7f0cac0a53e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0cc41018b0 tx=0x7f0cc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:54.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:54.912+0000 7f0cb9ffb640 1 -- 172.21.15.49:0/4209289106 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0cb404b020 con 0x7f0cc4101a50 2024-03-26T18:01:54.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:54 smithi115 ceph-mon[36174]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-03-26T18:01:54.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:54 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3024036574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:54.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:54 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4143862887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:54.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:54 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/159096735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:55.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.096+0000 7f0ccaec0640 1 -- 172.21.15.49:0/4209289106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cc402d420 con 0x7f0cc4101a50 2024-03-26T18:01:55.079 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.097+0000 7f0cb9ffb640 1 -- 172.21.15.49:0/4209289106 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24672 (secure 0 0 0) 0x7f0cb40bdba0 con 0x7f0cc4101a50 2024-03-26T18:01:55.080 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:55.080 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":94,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T18:01:45.470535+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:01:55.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.099+0000 7f0ccaec0640 1 -- 172.21.15.49:0/4209289106 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f0cac0a2f20 msgr2=0x7f0cac0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.099+0000 7f0ccaec0640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f0cac0a2f20 0x7f0cac0a53e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0cc41018b0 tx=0x7f0cc0009260 comp rx=0 tx=0).stop 2024-03-26T18:01:55.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.099+0000 7f0ccaec0640 1 -- 172.21.15.49:0/4209289106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc4101a50 msgr2=0x7f0cc419c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.099+0000 7f0ccaec0640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc4101a50 0x7f0cc419c910 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4005940 tx=0x7f0cb4032470 comp rx=0 tx=0).stop 2024-03-26T18:01:55.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.100+0000 7f0ccaec0640 1 -- 172.21.15.49:0/4209289106 shutdown_connections 2024-03-26T18:01:55.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.100+0000 7f0ccaec0640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f0cac0a2f20 0x7f0cac0a53e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.100+0000 7f0ccaec0640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4102410 0x7f0cc419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.100+0000 7f0ccaec0640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc4101a50 0x7f0cc419c910 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.100+0000 7f0ccaec0640 1 --2- 172.21.15.49:0/4209289106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc4100860 0x7f0cc406aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.100+0000 7f0ccaec0640 1 -- 172.21.15.49:0/4209289106 >> 172.21.15.49:0/4209289106 conn(0x7f0cc40fbdf0 msgr2=0x7f0cc4104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:55.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.100+0000 7f0ccaec0640 1 -- 172.21.15.49:0/4209289106 shutdown_connections 2024-03-26T18:01:55.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.100+0000 7f0ccaec0640 1 -- 172.21.15.49:0/4209289106 wait complete. 2024-03-26T18:01:55.093 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:59:45.382542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:59:46.221139+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T17:59:49.890383+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T18:01:55.094 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.223+0000 7f68386bf640 1 -- 172.21.15.49:0/4010183246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6830102360 msgr2=0x7f6830104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.223+0000 7f68386bf640 1 --2- 172.21.15.49:0/4010183246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6830102360 0x7f6830104750 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f682c009900 tx=0x7f682c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.223+0000 7f68386bf640 1 -- 172.21.15.49:0/4010183246 shutdown_connections 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.223+0000 7f68386bf640 1 --2- 172.21.15.49:0/4010183246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68301075c0 0x7f6830109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.223+0000 7f68386bf640 1 --2- 172.21.15.49:0/4010183246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6830104c90 0x7f6830107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.223+0000 7f68386bf640 1 --2- 172.21.15.49:0/4010183246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6830102360 0x7f6830104750 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.223+0000 7f68386bf640 1 -- 172.21.15.49:0/4010183246 >> 172.21.15.49:0/4010183246 conn(0x7f68300fbd70 msgr2=0x7f68300fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.223+0000 7f68386bf640 1 -- 172.21.15.49:0/4010183246 shutdown_connections 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.223+0000 7f68386bf640 1 -- 172.21.15.49:0/4010183246 wait complete. 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f68386bf640 1 Processor -- start 2024-03-26T18:01:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f68386bf640 1 -- start start 2024-03-26T18:01:55.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f68386bf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6830102360 0x7f6830101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f68386bf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6830104c90 0x7f6830100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f68386bf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68301075c0 0x7f6830100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f68386bf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f683010d560 con 0x7f6830104c90 2024-03-26T18:01:55.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f68386bf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f683010d3e0 con 0x7f6830102360 2024-03-26T18:01:55.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f68386bf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f683010d6e0 con 0x7f68301075c0 2024-03-26T18:01:55.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f6835c5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6830104c90 0x7f6830100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f683645b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6830102360 0x7f6830101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f6835c5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6830104c90 0x7f6830100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33180/0 (socket says 172.21.15.49:33180) 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f683645b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6830102360 0x7f6830101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48430/0 (socket says 172.21.15.49:48430) 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f6835c5a640 1 -- 172.21.15.49:0/2836556102 learned_addr learned my addr 172.21.15.49:0/2836556102 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f6836c5c640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68301075c0 0x7f6830100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f6835c5a640 1 -- 172.21.15.49:0/2836556102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68301075c0 msgr2=0x7f6830100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f6835c5a640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68301075c0 0x7f6830100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f6835c5a640 1 -- 172.21.15.49:0/2836556102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6830102360 msgr2=0x7f6830101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f6835c5a640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6830102360 0x7f6830101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.224+0000 7f6835c5a640 1 -- 172.21.15.49:0/2836556102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68301010e0 con 0x7f6830104c90 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.225+0000 7f6835c5a640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6830104c90 0x7f6830100310 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f682000e9a0 tx=0x7f682000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.225+0000 7f68277fe640 1 -- 172.21.15.49:0/2836556102 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682000cd80 con 0x7f6830104c90 2024-03-26T18:01:55.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.225+0000 7f68386bf640 1 -- 172.21.15.49:0/2836556102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68301a1d50 con 0x7f6830104c90 2024-03-26T18:01:55.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.225+0000 7f68277fe640 1 -- 172.21.15.49:0/2836556102 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68200035f0 con 0x7f6830104c90 2024-03-26T18:01:55.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.225+0000 7f68277fe640 1 -- 172.21.15.49:0/2836556102 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820010670 con 0x7f6830104c90 2024-03-26T18:01:55.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.225+0000 7f68386bf640 1 -- 172.21.15.49:0/2836556102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68301a2210 con 0x7f6830104c90 2024-03-26T18:01:55.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.226+0000 7f68386bf640 1 -- 172.21.15.49:0/2836556102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68300fd860 con 0x7f6830104c90 2024-03-26T18:01:55.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.228+0000 7f68277fe640 1 -- 172.21.15.49:0/2836556102 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f68200037a0 con 0x7f6830104c90 2024-03-26T18:01:55.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.228+0000 7f68277fe640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f681c0a2f20 0x7f681c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.229+0000 7f683645b640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f681c0a2f20 0x7f681c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.229+0000 7f68277fe640 1 -- 172.21.15.49:0/2836556102 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f68200cfb20 con 0x7f6830104c90 2024-03-26T18:01:55.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.229+0000 7f683645b640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f681c0a2f20 0x7f681c0a53e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f682c02c7b0 tx=0x7f682c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:55.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.233+0000 7f68277fe640 1 -- 172.21.15.49:0/2836556102 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f682009c600 con 0x7f6830104c90 2024-03-26T18:01:55.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.419+0000 7f68386bf640 1 -- 172.21.15.49:0/2836556102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f683002d490 con 0x7f6830104c90 2024-03-26T18:01:55.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.419+0000 7f68277fe640 1 -- 172.21.15.49:0/2836556102 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f682009bf00 con 0x7f6830104c90 2024-03-26T18:01:55.401 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T18:01:55.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 -- 172.21.15.49:0/2836556102 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f681c0a2f20 msgr2=0x7f681c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f681c0a2f20 0x7f681c0a53e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f682c02c7b0 tx=0x7f682c005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 -- 172.21.15.49:0/2836556102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6830104c90 msgr2=0x7f6830100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6830104c90 0x7f6830100310 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f682000e9a0 tx=0x7f682000ee60 comp rx=0 tx=0).stop 2024-03-26T18:01:55.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 -- 172.21.15.49:0/2836556102 shutdown_connections 2024-03-26T18:01:55.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f681c0a2f20 0x7f681c0a53e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f68301075c0 0x7f6830100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6830104c90 0x7f6830100310 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 --2- 172.21.15.49:0/2836556102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6830102360 0x7f6830101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 -- 172.21.15.49:0/2836556102 >> 172.21.15.49:0/2836556102 conn(0x7f68300fbd70 msgr2=0x7f6830102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:55.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 -- 172.21.15.49:0/2836556102 shutdown_connections 2024-03-26T18:01:55.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.421+0000 7f68386bf640 1 -- 172.21.15.49:0/2836556102 wait complete. 2024-03-26T18:01:55.414 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:01:55.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.544+0000 7fb34dea0640 1 -- 172.21.15.49:0/836026157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3481007e0 msgr2=0x7fb348100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.544+0000 7fb34dea0640 1 --2- 172.21.15.49:0/836026157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3481007e0 0x7fb348100c00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009900 tx=0x7fb33c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:55.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.544+0000 7fb34dea0640 1 -- 172.21.15.49:0/836026157 shutdown_connections 2024-03-26T18:01:55.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.544+0000 7fb34dea0640 1 --2- 172.21.15.49:0/836026157 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3481019d0 0x7fb348101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.544+0000 7fb34dea0640 1 --2- 172.21.15.49:0/836026157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb348102390 0x7fb348108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.544+0000 7fb34dea0640 1 --2- 172.21.15.49:0/836026157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3481007e0 0x7fb348100c00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.544+0000 7fb34dea0640 1 -- 172.21.15.49:0/836026157 >> 172.21.15.49:0/836026157 conn(0x7fb3480fbd70 msgr2=0x7fb3480fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:55.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.544+0000 7fb34dea0640 1 -- 172.21.15.49:0/836026157 shutdown_connections 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.544+0000 7fb34dea0640 1 -- 172.21.15.49:0/836026157 wait complete. 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.544+0000 7fb34dea0640 1 Processor -- start 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb34dea0640 1 -- start start 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb34dea0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3481007e0 0x7fb348197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb34dea0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3481019d0 0x7fb3481981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb34dea0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb348102390 0x7fb34819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb34dea0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb34810c450 con 0x7fb3481007e0 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb34dea0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fb34810c2d0 con 0x7fb348102390 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb34dea0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb34810c5d0 con 0x7fb3481019d0 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb3477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3481007e0 0x7fb348197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb346ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3481019d0 0x7fb3481981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb347fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb348102390 0x7fb34819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb3477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3481007e0 0x7fb348197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33192/0 (socket says 172.21.15.49:33192) 2024-03-26T18:01:55.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb3477fe640 1 -- 172.21.15.49:0/3739907384 learned_addr learned my addr 172.21.15.49:0/3739907384 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:55.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb3477fe640 1 -- 172.21.15.49:0/3739907384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3481019d0 msgr2=0x7fb3481981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb3477fe640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3481019d0 0x7fb3481981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb3477fe640 1 -- 172.21.15.49:0/3739907384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb348102390 msgr2=0x7fb34819f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb3477fe640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb348102390 0x7fb34819f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb3477fe640 1 -- 172.21.15.49:0/3739907384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34819f8b0 con 0x7fb3481007e0 2024-03-26T18:01:55.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb347fff640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb348102390 0x7fb34819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:55.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.545+0000 7fb346ffd640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3481019d0 0x7fb3481981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:55.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.546+0000 7fb3477fe640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3481007e0 0x7fb348197c70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb33c000c00 tx=0x7fb33c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:55.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.547+0000 7fb344ff9640 1 -- 172.21.15.49:0/3739907384 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c041070 con 0x7fb3481007e0 2024-03-26T18:01:55.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.547+0000 7fb34dea0640 1 -- 172.21.15.49:0/3739907384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34819fb40 con 0x7fb3481007e0 2024-03-26T18:01:55.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.547+0000 7fb344ff9640 1 -- 172.21.15.49:0/3739907384 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb33c004450 con 0x7fb3481007e0 2024-03-26T18:01:55.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.547+0000 7fb344ff9640 1 -- 172.21.15.49:0/3739907384 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c034440 con 0x7fb3481007e0 2024-03-26T18:01:55.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.547+0000 7fb34dea0640 1 -- 172.21.15.49:0/3739907384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3481a0080 con 0x7fb3481007e0 2024-03-26T18:01:55.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.548+0000 7fb34dea0640 1 -- 172.21.15.49:0/3739907384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb348068b00 con 0x7fb3481007e0 2024-03-26T18:01:55.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.549+0000 7fb344ff9640 1 -- 172.21.15.49:0/3739907384 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fb33c044070 con 0x7fb3481007e0 2024-03-26T18:01:55.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.549+0000 7fb344ff9640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3300a2f20 0x7fb3300a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.550+0000 7fb346ffd640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3300a2f20 0x7fb3300a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.550+0000 7fb344ff9640 1 -- 172.21.15.49:0/3739907384 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7fb33c0f1360 con 0x7fb3481007e0 2024-03-26T18:01:55.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.550+0000 7fb346ffd640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3300a2f20 0x7fb3300a53e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb348199170 tx=0x7fb338005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:55.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.555+0000 7fb344ff9640 1 -- 172.21.15.49:0/3739907384 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb33c0bd8f0 con 0x7fb3481007e0 2024-03-26T18:01:55.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.739+0000 7fb34dea0640 1 -- 172.21.15.49:0/3739907384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb3481a04f0 con 0x7fb3481007e0 2024-03-26T18:01:55.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.739+0000 7fb344ff9640 1 -- 172.21.15.49:0/3739907384 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fb33c0c1990 con 0x7fb3481007e0 2024-03-26T18:01:55.722 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:01:55.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 -- 172.21.15.49:0/3739907384 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3300a2f20 msgr2=0x7fb3300a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3300a2f20 0x7fb3300a53e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb348199170 tx=0x7fb338005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 -- 172.21.15.49:0/3739907384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3481007e0 msgr2=0x7fb348197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3481007e0 0x7fb348197c70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb33c000c00 tx=0x7fb33c02c640 comp rx=0 tx=0).stop 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 -- 172.21.15.49:0/3739907384 shutdown_connections 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fb3300a2f20 0x7fb3300a53e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3481019d0 0x7fb3481981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb348102390 0x7fb34819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 --2- 172.21.15.49:0/3739907384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3481007e0 0x7fb348197c70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 -- 172.21.15.49:0/3739907384 >> 172.21.15.49:0/3739907384 conn(0x7fb3480fbd70 msgr2=0x7fb3480fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 -- 172.21.15.49:0/3739907384 shutdown_connections 2024-03-26T18:01:55.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.741+0000 7fb34dea0640 1 -- 172.21.15.49:0/3739907384 wait complete. 2024-03-26T18:01:55.733 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:55 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2448140517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:55.733 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:55 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4209289106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:55.734 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:55 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2836556102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:55.735 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:01:55.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.864+0000 7f10ff9df640 1 -- 172.21.15.49:0/2699934325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f8102370 msgr2=0x7f10f8108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.864+0000 7f10ff9df640 1 --2- 172.21.15.49:0/2699934325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f8102370 0x7f10f8108c00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009ca0 tx=0x7f10f002c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.864+0000 7f10ff9df640 1 -- 172.21.15.49:0/2699934325 shutdown_connections 2024-03-26T18:01:55.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.864+0000 7f10ff9df640 1 --2- 172.21.15.49:0/2699934325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f8102370 0x7f10f8108c00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.864+0000 7f10ff9df640 1 --2- 172.21.15.49:0/2699934325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f81019b0 0x7f10f8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.864+0000 7f10ff9df640 1 --2- 172.21.15.49:0/2699934325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10f81007c0 0x7f10f8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.864+0000 7f10ff9df640 1 -- 172.21.15.49:0/2699934325 >> 172.21.15.49:0/2699934325 conn(0x7f10f80fbd50 msgr2=0x7f10f80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:55.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.864+0000 7f10ff9df640 1 -- 172.21.15.49:0/2699934325 shutdown_connections 2024-03-26T18:01:55.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.864+0000 7f10ff9df640 1 -- 172.21.15.49:0/2699934325 wait complete. 2024-03-26T18:01:55.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.864+0000 7f10ff9df640 1 Processor -- start 2024-03-26T18:01:55.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10ff9df640 1 -- start start 2024-03-26T18:01:55.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10ff9df640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f81007c0 0x7f10f8197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10ff9df640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10f81019b0 0x7f10f8198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10ff9df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f8102370 0x7f10f819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10ff9df640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f10f810c440 con 0x7f10f8102370 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10ff9df640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f10f810c2c0 con 0x7f10f81007c0 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10ff9df640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f10f810c5c0 con 0x7f10f81019b0 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fdf7c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f8102370 0x7f10f819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fdf7c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f8102370 0x7f10f819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33206/0 (socket says 172.21.15.49:33206) 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fd77b640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f81007c0 0x7f10f8197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fdf7c640 1 -- 172.21.15.49:0/553658439 learned_addr learned my addr 172.21.15.49:0/553658439 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fcf7a640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10f81019b0 0x7f10f8198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fdf7c640 1 -- 172.21.15.49:0/553658439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10f81019b0 msgr2=0x7f10f8198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fdf7c640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10f81019b0 0x7f10f8198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fdf7c640 1 -- 172.21.15.49:0/553658439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f81007c0 msgr2=0x7f10f8197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:55.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fdf7c640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f81007c0 0x7f10f8197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:55.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fdf7c640 1 -- 172.21.15.49:0/553658439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f819f800 con 0x7f10f8102370 2024-03-26T18:01:55.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fcf7a640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10f81019b0 0x7f10f8198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:55.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10fdf7c640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f8102370 0x7f10f819f190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f10f0037040 tx=0x7f10f0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:55.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.865+0000 7f10ee7fc640 1 -- 172.21.15.49:0/553658439 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0041020 con 0x7f10f8102370 2024-03-26T18:01:55.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.866+0000 7f10ee7fc640 1 -- 172.21.15.49:0/553658439 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f0034cd0 con 0x7f10f8102370 2024-03-26T18:01:55.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.866+0000 7f10ff9df640 1 -- 172.21.15.49:0/553658439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f819fa90 con 0x7f10f8102370 2024-03-26T18:01:55.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.866+0000 7f10ee7fc640 1 -- 172.21.15.49:0/553658439 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0039040 con 0x7f10f8102370 2024-03-26T18:01:55.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.866+0000 7f10ff9df640 1 -- 172.21.15.49:0/553658439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f819ff80 con 0x7f10f8102370 2024-03-26T18:01:55.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.867+0000 7f10ff9df640 1 -- 172.21.15.49:0/553658439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f81a01b0 con 0x7f10f8102370 2024-03-26T18:01:55.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.868+0000 7f10ee7fc640 1 -- 172.21.15.49:0/553658439 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f10f0044020 con 0x7f10f8102370 2024-03-26T18:01:55.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.868+0000 7f10ee7fc640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10e40a2f20 0x7f10e40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:55.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.868+0000 7f10fd77b640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10e40a2f20 0x7f10e40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:55.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.868+0000 7f10ee7fc640 1 -- 172.21.15.49:0/553658439 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f10f00f1630 con 0x7f10f8102370 2024-03-26T18:01:55.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.869+0000 7f10fd77b640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10e40a2f20 0x7f10e40a53e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f10f8101810 tx=0x7f10f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:55.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:55.873+0000 7f10ee7fc640 1 -- 172.21.15.49:0/553658439 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f10f004b020 con 0x7f10f8102370 2024-03-26T18:01:55.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:55 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2448140517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:55.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:55 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4209289106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:55.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:55 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2836556102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:55.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:55 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2448140517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:55.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:55 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4209289106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:55.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:55 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2836556102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:01:56.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.056+0000 7f10ff9df640 1 -- 172.21.15.49:0/553658439 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10f81a0380 con 0x7f10f8102370 2024-03-26T18:01:56.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.056+0000 7f10ee7fc640 1 -- 172.21.15.49:0/553658439 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f10f00be170 con 0x7f10f8102370 2024-03-26T18:01:56.039 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.058+0000 7f10ff9df640 1 -- 172.21.15.49:0/553658439 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10e40a2f20 msgr2=0x7f10e40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.058+0000 7f10ff9df640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10e40a2f20 0x7f10e40a53e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f10f8101810 tx=0x7f10f4005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.058+0000 7f10ff9df640 1 -- 172.21.15.49:0/553658439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f8102370 msgr2=0x7f10f819f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.058+0000 7f10ff9df640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f8102370 0x7f10f819f190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f10f0037040 tx=0x7f10f0032f40 comp rx=0 tx=0).stop 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.058+0000 7f10ff9df640 1 -- 172.21.15.49:0/553658439 shutdown_connections 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.058+0000 7f10ff9df640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f10e40a2f20 0x7f10e40a53e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.058+0000 7f10ff9df640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f10f81019b0 0x7f10f8198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.058+0000 7f10ff9df640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f10f8102370 0x7f10f819f190 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.058+0000 7f10ff9df640 1 --2- 172.21.15.49:0/553658439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10f81007c0 0x7f10f8197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.058+0000 7f10ff9df640 1 -- 172.21.15.49:0/553658439 >> 172.21.15.49:0/553658439 conn(0x7f10f80fbd50 msgr2=0x7f10f80fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.059+0000 7f10ff9df640 1 -- 172.21.15.49:0/553658439 shutdown_connections 2024-03-26T18:01:56.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.059+0000 7f10ff9df640 1 -- 172.21.15.49:0/553658439 wait complete. 2024-03-26T18:01:56.052 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T18:01:56.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 -- 172.21.15.49:0/1840570961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4102360 msgr2=0x7f12b4104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 --2- 172.21.15.49:0/1840570961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4102360 0x7f12b4104750 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009900 tx=0x7f12a402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:56.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 -- 172.21.15.49:0/1840570961 shutdown_connections 2024-03-26T18:01:56.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 --2- 172.21.15.49:0/1840570961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b4104c90 0x7f12b4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 --2- 172.21.15.49:0/1840570961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b41075c0 0x7f12b4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 --2- 172.21.15.49:0/1840570961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4102360 0x7f12b4104750 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 -- 172.21.15.49:0/1840570961 >> 172.21.15.49:0/1840570961 conn(0x7f12b40fbd70 msgr2=0x7f12b40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 -- 172.21.15.49:0/1840570961 shutdown_connections 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 -- 172.21.15.49:0/1840570961 wait complete. 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 Processor -- start 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.182+0000 7f12bc13b640 1 -- start start 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12bc13b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b4102360 0x7f12b4101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12bc13b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104c90 0x7f12b4100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12bc13b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b41075c0 0x7f12b4100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12bc13b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12b410d5e0 con 0x7f12b4104c90 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12bc13b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f12b410d460 con 0x7f12b41075c0 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12bc13b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f12b410d760 con 0x7f12b4102360 2024-03-26T18:01:56.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12b96d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104c90 0x7f12b4100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12b96d6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104c90 0x7f12b4100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33228/0 (socket says 172.21.15.49:33228) 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12b96d6640 1 -- 172.21.15.49:0/925033805 learned_addr learned my addr 172.21.15.49:0/925033805 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12ba6d8640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b41075c0 0x7f12b4100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12b9ed7640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b4102360 0x7f12b4101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12b96d6640 1 -- 172.21.15.49:0/925033805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b4102360 msgr2=0x7f12b4101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12b96d6640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b4102360 0x7f12b4101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12b96d6640 1 -- 172.21.15.49:0/925033805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b41075c0 msgr2=0x7f12b4100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12b96d6640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b41075c0 0x7f12b4100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12b96d6640 1 -- 172.21.15.49:0/925033805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b41010e0 con 0x7f12b4104c90 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12b96d6640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104c90 0x7f12b4100310 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f12b000e9f0 tx=0x7f12b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12aaffd640 1 -- 172.21.15.49:0/925033805 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b000ccc0 con 0x7f12b4104c90 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12aaffd640 1 -- 172.21.15.49:0/925033805 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b0005500 con 0x7f12b4104c90 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12bc13b640 1 -- 172.21.15.49:0/925033805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b41a1d80 con 0x7f12b4104c90 2024-03-26T18:01:56.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12aaffd640 1 -- 172.21.15.49:0/925033805 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0010690 con 0x7f12b4104c90 2024-03-26T18:01:56.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.183+0000 7f12bc13b640 1 -- 172.21.15.49:0/925033805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b41a21c0 con 0x7f12b4104c90 2024-03-26T18:01:56.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.184+0000 7f12bc13b640 1 -- 172.21.15.49:0/925033805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b40fd860 con 0x7f12b4104c90 2024-03-26T18:01:56.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.186+0000 7f12aaffd640 1 -- 172.21.15.49:0/925033805 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f12b00046c0 con 0x7f12b4104c90 2024-03-26T18:01:56.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.186+0000 7f12aaffd640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12a00a2f20 0x7f12a00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.186+0000 7f12aaffd640 1 -- 172.21.15.49:0/925033805 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f12b00cfcd0 con 0x7f12b4104c90 2024-03-26T18:01:56.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.186+0000 7f12b9ed7640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12a00a2f20 0x7f12a00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.187+0000 7f12b9ed7640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12a00a2f20 0x7f12a00a53e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f12a402c240 tx=0x7f12a402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:56.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.190+0000 7f12aaffd640 1 -- 172.21.15.49:0/925033805 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f12b0053740 con 0x7f12b4104c90 2024-03-26T18:01:56.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.375+0000 7f12bc13b640 1 -- 172.21.15.49:0/925033805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f12b402d490 con 0x7f12b4104c90 2024-03-26T18:01:56.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.375+0000 7f12aaffd640 1 -- 172.21.15.49:0/925033805 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f12b009c780 con 0x7f12b4104c90 2024-03-26T18:01:56.358 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.377+0000 7f12bc13b640 1 -- 172.21.15.49:0/925033805 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12a00a2f20 msgr2=0x7f12a00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.377+0000 7f12bc13b640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12a00a2f20 0x7f12a00a53e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f12a402c240 tx=0x7f12a402df90 comp rx=0 tx=0).stop 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.377+0000 7f12bc13b640 1 -- 172.21.15.49:0/925033805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104c90 msgr2=0x7f12b4100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.377+0000 7f12bc13b640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104c90 0x7f12b4100310 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f12b000e9f0 tx=0x7f12b000edf0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.377+0000 7f12bc13b640 1 -- 172.21.15.49:0/925033805 shutdown_connections 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.377+0000 7f12bc13b640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f12a00a2f20 0x7f12a00a53e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.377+0000 7f12bc13b640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12b41075c0 0x7f12b4100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.378+0000 7f12bc13b640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12b4104c90 0x7f12b4100310 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.378+0000 7f12bc13b640 1 --2- 172.21.15.49:0/925033805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f12b4102360 0x7f12b4101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.378+0000 7f12bc13b640 1 -- 172.21.15.49:0/925033805 >> 172.21.15.49:0/925033805 conn(0x7f12b40fbd70 msgr2=0x7f12b4102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.378+0000 7f12bc13b640 1 -- 172.21.15.49:0/925033805 shutdown_connections 2024-03-26T18:01:56.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.378+0000 7f12bc13b640 1 -- 172.21.15.49:0/925033805 wait complete. 2024-03-26T18:01:56.371 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:01:56.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.497+0000 7f7fd18cc640 1 -- 172.21.15.49:0/3172442623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fcc102ae0 msgr2=0x7f7fcc102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.497+0000 7f7fd18cc640 1 --2- 172.21.15.49:0/3172442623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fcc102ae0 0x7f7fcc102f60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009900 tx=0x7f7fb402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.497+0000 7f7fd18cc640 1 -- 172.21.15.49:0/3172442623 shutdown_connections 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.497+0000 7f7fd18cc640 1 --2- 172.21.15.49:0/3172442623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fcc1034a0 0x7f7fcc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.497+0000 7f7fd18cc640 1 --2- 172.21.15.49:0/3172442623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fcc102ae0 0x7f7fcc102f60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.497+0000 7f7fd18cc640 1 --2- 172.21.15.49:0/3172442623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fcc1018f0 0x7f7fcc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.497+0000 7f7fd18cc640 1 -- 172.21.15.49:0/3172442623 >> 172.21.15.49:0/3172442623 conn(0x7f7fcc0fcea0 msgr2=0x7f7fcc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.497+0000 7f7fd18cc640 1 -- 172.21.15.49:0/3172442623 shutdown_connections 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.497+0000 7f7fd18cc640 1 -- 172.21.15.49:0/3172442623 wait complete. 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fd18cc640 1 Processor -- start 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fd18cc640 1 -- start start 2024-03-26T18:01:56.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fd18cc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fcc1018f0 0x7f7fcc197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fd18cc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fcc102ae0 0x7f7fcc1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fd18cc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fcc1034a0 0x7f7fcc19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fd18cc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10d570 con 0x7f7fcc102ae0 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fd18cc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10d3f0 con 0x7f7fcc1034a0 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fd18cc640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10d6f0 con 0x7f7fcc1018f0 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fc27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fcc102ae0 0x7f7fcc1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fc27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fcc102ae0 0x7f7fcc1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33254/0 (socket says 172.21.15.49:33254) 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fcc1034a0 0x7f7fcc19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fcaffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fcc1018f0 0x7f7fcc197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.498+0000 7f7fc27fc640 1 -- 172.21.15.49:0/1245648011 learned_addr learned my addr 172.21.15.49:0/1245648011 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fc27fc640 1 -- 172.21.15.49:0/1245648011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fcc1018f0 msgr2=0x7f7fcc197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fc27fc640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fcc1018f0 0x7f7fcc197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fc27fc640 1 -- 172.21.15.49:0/1245648011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fcc1034a0 msgr2=0x7f7fcc19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fc27fc640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fcc1034a0 0x7f7fcc19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fc27fc640 1 -- 172.21.15.49:0/1245648011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc19f8c0 con 0x7f7fcc102ae0 2024-03-26T18:01:56.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fcb7fe640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fcc1034a0 0x7f7fcc19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:56.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fc27fc640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fcc102ae0 0x7f7fcc1981c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009230 tx=0x7f7fb402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:56.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fd08ca640 1 -- 172.21.15.49:0/1245648011 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4041070 con 0x7f7fcc102ae0 2024-03-26T18:01:56.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fd08ca640 1 -- 172.21.15.49:0/1245648011 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb4004450 con 0x7f7fcc102ae0 2024-03-26T18:01:56.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fd18cc640 1 -- 172.21.15.49:0/1245648011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc19fb50 con 0x7f7fcc102ae0 2024-03-26T18:01:56.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fd08ca640 1 -- 172.21.15.49:0/1245648011 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4003db0 con 0x7f7fcc102ae0 2024-03-26T18:01:56.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.499+0000 7f7fd18cc640 1 -- 172.21.15.49:0/1245648011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc1a0040 con 0x7f7fcc102ae0 2024-03-26T18:01:56.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.500+0000 7f7fd18cc640 1 -- 172.21.15.49:0/1245648011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fcc108c10 con 0x7f7fcc102ae0 2024-03-26T18:01:56.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.501+0000 7f7fd08ca640 1 -- 172.21.15.49:0/1245648011 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f7fb4044070 con 0x7f7fcc102ae0 2024-03-26T18:01:56.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.502+0000 7f7fd08ca640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7fb80a2f20 0x7f7fb80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.502+0000 7f7fcaffd640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7fb80a2f20 0x7f7fb80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.502+0000 7f7fd08ca640 1 -- 172.21.15.49:0/1245648011 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f7fb40f12c0 con 0x7f7fcc102ae0 2024-03-26T18:01:56.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.502+0000 7f7fcaffd640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7fb80a2f20 0x7f7fb80a53e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc198b50 tx=0x7f7fbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:56.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.506+0000 7f7fd08ca640 1 -- 172.21.15.49:0/1245648011 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7fb40bdd70 con 0x7f7fcc102ae0 2024-03-26T18:01:56.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.731+0000 7f7fd18cc640 1 -- 172.21.15.49:0/1245648011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fcc0624f0 con 0x7f7fcc102ae0 2024-03-26T18:01:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.732+0000 7f7fd08ca640 1 -- 172.21.15.49:0/1245648011 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11943 (secure 0 0 0) 0x7f7fb40bd670 con 0x7f7fcc102ae0 2024-03-26T18:01:56.714 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:56.714 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:59:46.796381+0000","modified":"2024-03-26T18:00:09.450708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24503,"mds_2":24478,"mds_3":24428,"mds_4":24388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24478,24503,24388,24428,24533]},"id":1}]} 2024-03-26T18:01:56.714 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2024-03-26T18:01:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.733+0000 7f7fd18cc640 1 -- 172.21.15.49:0/1245648011 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7fb80a2f20 msgr2=0x7f7fb80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.733+0000 7f7fd18cc640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7fb80a2f20 0x7f7fb80a53e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc198b50 tx=0x7f7fbc005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.733+0000 7f7fd18cc640 1 -- 172.21.15.49:0/1245648011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fcc102ae0 msgr2=0x7f7fcc1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.733+0000 7f7fd18cc640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fcc102ae0 0x7f7fcc1981c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009230 tx=0x7f7fb402c650 comp rx=0 tx=0).stop 2024-03-26T18:01:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.734+0000 7f7fd18cc640 1 -- 172.21.15.49:0/1245648011 shutdown_connections 2024-03-26T18:01:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.734+0000 7f7fd18cc640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f7fb80a2f20 0x7f7fb80a53e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.734+0000 7f7fd18cc640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fcc1034a0 0x7f7fcc19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.734+0000 7f7fd18cc640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fcc102ae0 0x7f7fcc1981c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.734+0000 7f7fd18cc640 1 --2- 172.21.15.49:0/1245648011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fcc1018f0 0x7f7fcc197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.734+0000 7f7fd18cc640 1 -- 172.21.15.49:0/1245648011 >> 172.21.15.49:0/1245648011 conn(0x7f7fcc0fcea0 msgr2=0x7f7fcc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.734+0000 7f7fd18cc640 1 -- 172.21.15.49:0/1245648011 shutdown_connections 2024-03-26T18:01:56.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.734+0000 7f7fd18cc640 1 -- 172.21.15.49:0/1245648011 wait complete. 2024-03-26T18:01:56.726 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:56 smithi049 ceph-mon[32331]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:01:56.726 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:56 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3739907384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:56.726 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:56 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/553658439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:56.726 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:56 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/925033805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:56.727 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:01:56.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.853+0000 7f56689e6640 1 -- 172.21.15.49:0/3869952247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5664107640 msgr2=0x7f5664109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.853+0000 7f56689e6640 1 --2- 172.21.15.49:0/3869952247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5664107640 0x7f5664109a90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f565c009ca0 tx=0x7f565c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.853+0000 7f56689e6640 1 -- 172.21.15.49:0/3869952247 shutdown_connections 2024-03-26T18:01:56.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.853+0000 7f56689e6640 1 --2- 172.21.15.49:0/3869952247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5664104d10 0x7f5664107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.853+0000 7f56689e6640 1 --2- 172.21.15.49:0/3869952247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5664107640 0x7f5664109a90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.853+0000 7f56689e6640 1 --2- 172.21.15.49:0/3869952247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56641023e0 0x7f56641047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.853+0000 7f56689e6640 1 -- 172.21.15.49:0/3869952247 >> 172.21.15.49:0/3869952247 conn(0x7f56640fbdf0 msgr2=0x7f56640fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:56.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.853+0000 7f56689e6640 1 -- 172.21.15.49:0/3869952247 shutdown_connections 2024-03-26T18:01:56.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.853+0000 7f56689e6640 1 -- 172.21.15.49:0/3869952247 wait complete. 2024-03-26T18:01:56.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f56689e6640 1 Processor -- start 2024-03-26T18:01:56.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f56689e6640 1 -- start start 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f56689e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56641023e0 0x7f5664101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f56689e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5664104d10 0x7f5664100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f56689e6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5664107640 0x7f56641008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f56689e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f566410d570 con 0x7f5664104d10 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f56689e6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f566410d3f0 con 0x7f56641023e0 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f56689e6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f566410d6f0 con 0x7f5664107640 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f565bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5664104d10 0x7f5664100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f565bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5664104d10 0x7f5664100390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33268/0 (socket says 172.21.15.49:33268) 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f566259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56641023e0 0x7f5664101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f565bfff640 1 -- 172.21.15.49:0/824940072 learned_addr learned my addr 172.21.15.49:0/824940072 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f5662d9d640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5664107640 0x7f56641008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f565bfff640 1 -- 172.21.15.49:0/824940072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5664107640 msgr2=0x7f56641008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f565bfff640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5664107640 0x7f56641008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f565bfff640 1 -- 172.21.15.49:0/824940072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56641023e0 msgr2=0x7f5664101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f565bfff640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56641023e0 0x7f5664101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f565bfff640 1 -- 172.21.15.49:0/824940072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56641010f0 con 0x7f5664104d10 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.854+0000 7f566259c640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56641023e0 0x7f5664101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.855+0000 7f565bfff640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5664104d10 0x7f5664100390 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f564c00e9c0 tx=0x7f564c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.855+0000 7f565b7fe640 1 -- 172.21.15.49:0/824940072 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c00cd80 con 0x7f5664104d10 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.855+0000 7f565b7fe640 1 -- 172.21.15.49:0/824940072 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c005500 con 0x7f5664104d10 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.855+0000 7f56689e6640 1 -- 172.21.15.49:0/824940072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56641a1cd0 con 0x7f5664104d10 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.855+0000 7f565b7fe640 1 -- 172.21.15.49:0/824940072 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c010690 con 0x7f5664104d10 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.855+0000 7f56689e6640 1 -- 172.21.15.49:0/824940072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56641a21d0 con 0x7f5664104d10 2024-03-26T18:01:56.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.856+0000 7f56689e6640 1 -- 172.21.15.49:0/824940072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56640fd8e0 con 0x7f5664104d10 2024-03-26T18:01:56.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.857+0000 7f565b7fe640 1 -- 172.21.15.49:0/824940072 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f564c0046c0 con 0x7f5664104d10 2024-03-26T18:01:56.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.858+0000 7f565b7fe640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56500a2f20 0x7f56500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:56.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.858+0000 7f566259c640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56500a2f20 0x7f56500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:56.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.858+0000 7f565b7fe640 1 -- 172.21.15.49:0/824940072 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f564c0cfcd0 con 0x7f5664104d10 2024-03-26T18:01:56.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.858+0000 7f566259c640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56500a2f20 0x7f56500a53e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f56540095f0 tx=0x7f5654005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:56.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:56.861+0000 7f565b7fe640 1 -- 172.21.15.49:0/824940072 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f564c053740 con 0x7f5664104d10 2024-03-26T18:01:56.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:56 smithi171 ceph-mon[36033]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:01:56.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:56 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3739907384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:56.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:56 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/553658439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:56.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:56 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/925033805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:56.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:56 smithi115 ceph-mon[36174]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:01:56.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:56 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3739907384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:01:56.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:56 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/553658439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:01:56.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:56 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/925033805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:01:57.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.045+0000 7f56689e6640 1 -- 172.21.15.49:0/824940072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f566402d3c0 con 0x7f5664104d10 2024-03-26T18:01:57.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.046+0000 7f565b7fe640 1 -- 172.21.15.49:0/824940072 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24672 (secure 0 0 0) 0x7f564c09c780 con 0x7f5664104d10 2024-03-26T18:01:57.028 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:01:57.029 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":94,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T18:01:45.470535+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 -- 172.21.15.49:0/824940072 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56500a2f20 msgr2=0x7f56500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56500a2f20 0x7f56500a53e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f56540095f0 tx=0x7f5654005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 -- 172.21.15.49:0/824940072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5664104d10 msgr2=0x7f5664100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5664104d10 0x7f5664100390 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f564c00e9c0 tx=0x7f564c00ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 -- 172.21.15.49:0/824940072 shutdown_connections 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56500a2f20 0x7f56500a53e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5664107640 0x7f56641008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5664104d10 0x7f5664100390 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 --2- 172.21.15.49:0/824940072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56641023e0 0x7f5664101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 -- 172.21.15.49:0/824940072 >> 172.21.15.49:0/824940072 conn(0x7f56640fbdf0 msgr2=0x7f5664102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 -- 172.21.15.49:0/824940072 shutdown_connections 2024-03-26T18:01:57.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.048+0000 7f56689e6640 1 -- 172.21.15.49:0/824940072 wait complete. 2024-03-26T18:01:57.042 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T17:59:46.796381+0000', 'modified': '2024-03-26T18:00:09.450708+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24533, 'mds_1': 24503, 'mds_2': 24478, 'mds_3': 24428, 'mds_4': 24388}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6839/906046850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 906046850}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 906046850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6841/935071179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6840', 'nonce': 935071179}, {'type': 'v1', 'addr': '172.21.15.49:6841', 'nonce': 935071179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24388': {'gid': 24388, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.171:6833/2067087026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6832', 'nonce': 2067087026}, {'type': 'v1', 'addr': '172.21.15.171:6833', 'nonce': 2067087026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.115:6833/3604276420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 3604276420}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 3604276420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6835/3854985421', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 3854985421}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 3854985421}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/2428288879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 2428288879}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 2428288879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/4085026954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 4085026954}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 4085026954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.171:6839/1082475840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 1082475840}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 1082475840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.115:6837/1993258438', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 1993258438}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 1993258438}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6839/1435040687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 1435040687}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 1435040687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24533, 'qdb_cluster': [24478, 24503, 24388, 24428, 24533]} 2024-03-26T18:01:57.042 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-26T18:01:57.043 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-03-26T18:01:57.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.170+0000 7fa3842ea640 1 -- 172.21.15.49:0/3767463481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa37c102360 msgr2=0x7fa37c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.170+0000 7fa3842ea640 1 --2- 172.21.15.49:0/3767463481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa37c102360 0x7fa37c104750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa378009900 tx=0x7fa37802bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.170+0000 7fa3842ea640 1 -- 172.21.15.49:0/3767463481 shutdown_connections 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.170+0000 7fa3842ea640 1 --2- 172.21.15.49:0/3767463481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa37c1075c0 0x7fa37c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.170+0000 7fa3842ea640 1 --2- 172.21.15.49:0/3767463481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c104c90 0x7fa37c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.170+0000 7fa3842ea640 1 --2- 172.21.15.49:0/3767463481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa37c102360 0x7fa37c104750 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.170+0000 7fa3842ea640 1 -- 172.21.15.49:0/3767463481 >> 172.21.15.49:0/3767463481 conn(0x7fa37c0fbd70 msgr2=0x7fa37c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.170+0000 7fa3842ea640 1 -- 172.21.15.49:0/3767463481 shutdown_connections 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.170+0000 7fa3842ea640 1 -- 172.21.15.49:0/3767463481 wait complete. 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa3842ea640 1 Processor -- start 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa3842ea640 1 -- start start 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa3842ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa37c102360 0x7fa37c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa3842ea640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa37c104c90 0x7fa37c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa3842ea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c1075c0 0x7fa37c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa3842ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c10d560 con 0x7fa37c102360 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa3842ea640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c10d3e0 con 0x7fa37c1075c0 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa3842ea640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c10d6e0 con 0x7fa37c104c90 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa382086640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa37c102360 0x7fa37c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa382086640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa37c102360 0x7fa37c197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33298/0 (socket says 172.21.15.49:33298) 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa382086640 1 -- 172.21.15.49:0/4000704534 learned_addr learned my addr 172.21.15.49:0/4000704534 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa381885640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa37c104c90 0x7fa37c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa382887640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c1075c0 0x7fa37c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa382086640 1 -- 172.21.15.49:0/4000704534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa37c104c90 msgr2=0x7fa37c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa382086640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa37c104c90 0x7fa37c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa382086640 1 -- 172.21.15.49:0/4000704534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c1075c0 msgr2=0x7fa37c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa382086640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c1075c0 0x7fa37c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.171+0000 7fa382086640 1 -- 172.21.15.49:0/4000704534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c19f860 con 0x7fa37c102360 2024-03-26T18:01:57.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.172+0000 7fa382086640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa37c102360 0x7fa37c197c50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa378009230 tx=0x7fa37802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.172+0000 7fa3737fe640 1 -- 172.21.15.49:0/4000704534 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378041070 con 0x7fa37c102360 2024-03-26T18:01:57.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.172+0000 7fa3737fe640 1 -- 172.21.15.49:0/4000704534 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa378004450 con 0x7fa37c102360 2024-03-26T18:01:57.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.172+0000 7fa3842ea640 1 -- 172.21.15.49:0/4000704534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c19faf0 con 0x7fa37c102360 2024-03-26T18:01:57.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.172+0000 7fa3737fe640 1 -- 172.21.15.49:0/4000704534 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378003e20 con 0x7fa37c102360 2024-03-26T18:01:57.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.172+0000 7fa3842ea640 1 -- 172.21.15.49:0/4000704534 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c19ffe0 con 0x7fa37c102360 2024-03-26T18:01:57.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.174+0000 7fa3737fe640 1 -- 172.21.15.49:0/4000704534 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7fa378044070 con 0x7fa37c102360 2024-03-26T18:01:57.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.174+0000 7fa386fb1740 1 Processor -- start 2024-03-26T18:01:57.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.174+0000 7fa386fb1740 1 -- start start 2024-03-26T18:01:57.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x565373919470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 0x56537391a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x565373921470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa386fb1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565373879270 con 0x56537391abe0 2024-03-26T18:01:57.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa386fb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x565373821c80 con 0x56537391a220 2024-03-26T18:01:57.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa386fb1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565373875220 con 0x565373919050 2024-03-26T18:01:57.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa3737fe640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa3680a2f20 0x7fa3680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa381885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x565373921470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa382887640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x565373919470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa382086640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 0x56537391a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa382887640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x565373919470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:36704/0 (socket says 172.21.15.49:36704) 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa381885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x565373921470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33312/0 (socket says 172.21.15.49:33312) 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa381885640 1 -- 172.21.15.49:0/1922527928 learned_addr learned my addr 172.21.15.49:0/1922527928 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa382887640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa3680a2f20 0x7fa3680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa3737fe640 1 -- 172.21.15.49:0/4000704534 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7fa3780f1360 con 0x7fa37c102360 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa381885640 1 -- 172.21.15.49:0/1922527928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 msgr2=0x565373919470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa381885640 1 --2- 172.21.15.49:0/1922527928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x565373919470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa381885640 1 -- 172.21.15.49:0/1922527928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 msgr2=0x56537391a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa381885640 1 --2- 172.21.15.49:0/1922527928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 0x56537391a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa381885640 1 -- 172.21.15.49:0/1922527928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565373833fb0 con 0x56537391abe0 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa382086640 1 --2- 172.21.15.49:0/1922527928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 0x56537391a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa381885640 1 --2- 172.21.15.49:0/1922527928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x565373921470 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009c90 tx=0x7fa36c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c98777a0936b8b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.175+0000 7fa382887640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa3680a2f20 0x7fa3680a53e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x56537391a120 tx=0x7fa374009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa370bfe640 1 -- 172.21.15.49:0/1922527928 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c01e070 con 0x56537391abe0 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa370bfe640 1 -- 172.21.15.49:0/1922527928 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36c019b10 con 0x56537391abe0 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa370bfe640 1 -- 172.21.15.49:0/1922527928 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c00eaf0 con 0x56537391abe0 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa386fb1740 1 -- 172.21.15.49:0/1922527928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 msgr2=0x565373921470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa386fb1740 1 --2- 172.21.15.49:0/1922527928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x565373921470 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009c90 tx=0x7fa36c011810 comp rx=0 tx=0).stop 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa386fb1740 1 -- 172.21.15.49:0/1922527928 shutdown_connections 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa386fb1740 1 --2- 172.21.15.49:0/1922527928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x565373921470 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa386fb1740 1 --2- 172.21.15.49:0/1922527928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 0x56537391a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa386fb1740 1 --2- 172.21.15.49:0/1922527928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x565373919470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa386fb1740 1 -- 172.21.15.49:0/1922527928 >> 172.21.15.49:0/1922527928 conn(0x565373910030 msgr2=0x565373910430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa386fb1740 1 -- 172.21.15.49:0/1922527928 shutdown_connections 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.176+0000 7fa386fb1740 1 -- 172.21.15.49:0/1922527928 wait complete. 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa386fb1740 1 -- start start 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa386fb1740 1 Processor -- start 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x5653739235c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 0x565373923b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x56537392ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa386fb1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565373921bf0 con 0x56537391abe0 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa386fb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x565373834fc0 con 0x56537391a220 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa386fb1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565373921d70 con 0x565373919050 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa386fb1740 10 client.0 _ll_register_callbacks cb 0x56537382f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa382887640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x56537392ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa382887640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x56537392ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33326/0 (socket says 172.21.15.49:33326) 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa382887640 1 -- 172.21.15.49:0/4052119391 learned_addr learned my addr 172.21.15.49:0/4052119391 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa382086640 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x5653739235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa381885640 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 0x565373923b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa382887640 1 -- 172.21.15.49:0/4052119391 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 msgr2=0x5653739235c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa382887640 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x5653739235c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa382887640 1 -- 172.21.15.49:0/4052119391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 msgr2=0x565373923b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.177+0000 7fa382887640 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 0x565373923b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa382887640 1 -- 172.21.15.49:0/4052119391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537358d2d0 con 0x56537391abe0 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa382887640 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x56537392ab40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa374011ee0 tx=0x7fa374004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42e8dcb6806e556 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa365ffb640 1 -- 172.21.15.49:0/4052119391 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37401e410 con 0x56537391abe0 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa365ffb640 1 -- 172.21.15.49:0/4052119391 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7fa37401e5b0 con 0x56537391abe0 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa386fb1740 1 -- 172.21.15.49:0/4052119391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa36c019b10 con 0x56537391abe0 2024-03-26T18:01:57.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa365ffb640 1 -- 172.21.15.49:0/4052119391 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa374035050 con 0x56537391abe0 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa386fb1740 10 client.15420 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa386fb1740 1 -- 172.21.15.49:0/4052119391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5653736576b0 con 0x56537391abe0 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa365ffb640 1 -- 172.21.15.49:0/4052119391 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36c019b10 con 0x56537391abe0 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa365ffb640 1 -- 172.21.15.49:0/4052119391 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa37401d070 con 0x56537391abe0 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa386fb1740 10 client.15420 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa386fb1740 10 client.15420 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa386fb1740 20 client.15420 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa386fb1740 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x565373930480 0x5653739318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.178+0000 7fa386fb1740 4 client.15420 mds_command: new command op to 24388 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.179+0000 7fa386fb1740 1 -- 172.21.15.49:0/4052119391 --> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653738867b0 con 0x565373930480 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.179+0000 7fa382086640 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x565373930480 0x5653739318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.179+0000 7fa382086640 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x565373930480 0x5653739318f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1b4af9624149db9f server_cookie=ef1c48561734b243 in_seq=0 out_seq=0 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.179+0000 7fa365ffb640 10 client.15420 ms_handle_connect on v2:172.21.15.171:6832/2067087026 2024-03-26T18:01:57.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa365ffb640 1 -- 172.21.15.49:0/4052119391 <== mds.4 v2:172.21.15.171:6832/2067087026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5653738867b0 con 0x565373930480 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa365ffb640 10 client.15420 handle_command_reply: tid=0 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 client.15420 shutdown 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 -- 172.21.15.49:0/4052119391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 msgr2=0x56537392ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x56537392ab40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa374011ee0 tx=0x7fa374004ab0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 -- 172.21.15.49:0/4052119391 shutdown_connections 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x565373930480 0x5653739318f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391abe0 0x56537392ab40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391a220 0x565373923b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa365ffb640 0 client.15420 ms_handle_reset on v2:172.21.15.171:6832/2067087026 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 --2- 172.21.15.49:0/4052119391 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x5653739235c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 -- 172.21.15.49:0/4052119391 >> 172.21.15.49:0/4052119391 conn(0x565373910030 msgr2=0x56537390f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 -- 172.21.15.49:0/4052119391 shutdown_connections 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 1 -- 172.21.15.49:0/4052119391 wait complete. 2024-03-26T18:01:57.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.180+0000 7fa386fb1740 20 client.15420 trim_cache size 0 max 16384 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.198+0000 7fa386fb1740 1 Processor -- start 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.198+0000 7fa386fb1740 1 -- start start 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.198+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565373919050 0x5653739235c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56537391a220 0x565373923b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 0x56537392aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa386fb1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x56537392fc30 con 0x565373919050 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa386fb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x565373930300 con 0x56537391abe0 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa386fb1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5653739309d0 con 0x56537391a220 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382086640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565373919050 0x5653739235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382086640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565373919050 0x5653739235c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33328/0 (socket says 172.21.15.49:33328) 2024-03-26T18:01:57.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382887640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 0x56537392aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382086640 1 -- 172.21.15.49:0/1594465505 learned_addr learned my addr 172.21.15.49:0/1594465505 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa381885640 1 --2- 172.21.15.49:0/1594465505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56537391a220 0x565373923b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382086640 1 -- 172.21.15.49:0/1594465505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56537391a220 msgr2=0x565373923b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382086640 1 --2- 172.21.15.49:0/1594465505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56537391a220 0x565373923b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382086640 1 -- 172.21.15.49:0/1594465505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 msgr2=0x56537392aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382086640 1 --2- 172.21.15.49:0/1594465505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 0x56537392aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382086640 1 -- 172.21.15.49:0/1594465505 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565373919ca0 con 0x565373919050 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382887640 1 --2- 172.21.15.49:0/1594465505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 0x56537392aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa381885640 1 --2- 172.21.15.49:0/1594465505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56537391a220 0x565373923b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa382086640 1 --2- 172.21.15.49:0/1594465505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565373919050 0x5653739235c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa378000c00 tx=0x7fa378088280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3d000eaaaa10fb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa367fff640 1 -- 172.21.15.49:0/1594465505 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37804b040 con 0x565373919050 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa367fff640 1 -- 172.21.15.49:0/1594465505 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa378004c60 con 0x565373919050 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.199+0000 7fa367fff640 1 -- 172.21.15.49:0/1594465505 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3780c1860 con 0x565373919050 2024-03-26T18:01:57.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 -- 172.21.15.49:0/1594465505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565373919050 msgr2=0x5653739235c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 --2- 172.21.15.49:0/1594465505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565373919050 0x5653739235c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa378000c00 tx=0x7fa378088280 comp rx=0 tx=0).stop 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 -- 172.21.15.49:0/1594465505 shutdown_connections 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 --2- 172.21.15.49:0/1594465505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56537391a220 0x565373923b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 --2- 172.21.15.49:0/1594465505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 0x56537392aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 --2- 172.21.15.49:0/1594465505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x565373919050 0x5653739235c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 -- 172.21.15.49:0/1594465505 >> 172.21.15.49:0/1594465505 conn(0x565373910030 msgr2=0x565373910410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 -- 172.21.15.49:0/1594465505 shutdown_connections 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 -- 172.21.15.49:0/1594465505 wait complete. 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 -- start start 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 Processor -- start 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x565373926390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391a220 0x5653739268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 0x5653739298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x565373931770 con 0x56537391a220 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5653739310a0 con 0x56537391abe0 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.200+0000 7fa386fb1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565373931e40 con 0x565373919050 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa381885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391a220 0x5653739268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa386fb1740 10 client.0 _ll_register_callbacks cb 0x7fa374011da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa381885640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391a220 0x5653739268d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33340/0 (socket says 172.21.15.49:33340) 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa381885640 1 -- 172.21.15.49:0/2363862887 learned_addr learned my addr 172.21.15.49:0/2363862887 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa382086640 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x565373926390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa382887640 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 0x5653739298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa381885640 1 -- 172.21.15.49:0/2363862887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 msgr2=0x565373926390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa381885640 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x565373926390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa381885640 1 -- 172.21.15.49:0/2363862887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 msgr2=0x5653739298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa381885640 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 0x5653739298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa381885640 1 -- 172.21.15.49:0/2363862887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565373929ee0 con 0x56537391a220 2024-03-26T18:01:57.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa381885640 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391a220 0x5653739268d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa36c01c040 tx=0x7fa36c009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec16a469d4cd7ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa365ffb640 1 -- 172.21.15.49:0/2363862887 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c029070 con 0x56537391a220 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa365ffb640 1 -- 172.21.15.49:0/2363862887 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7fa36c005240 con 0x56537391a220 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa386fb1740 1 -- 172.21.15.49:0/2363862887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa378004c60 con 0x56537391a220 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa365ffb640 1 -- 172.21.15.49:0/2363862887 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa36c035050 con 0x56537391a220 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa386fb1740 10 client.15426 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.201+0000 7fa386fb1740 1 -- 172.21.15.49:0/2363862887 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565373996350 con 0x56537391a220 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa365ffb640 1 -- 172.21.15.49:0/2363862887 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa378004c60 con 0x56537391a220 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa365ffb640 1 -- 172.21.15.49:0/2363862887 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa36c025460 con 0x56537391a220 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa386fb1740 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa386fb1740 10 client.15426 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa386fb1740 20 client.15426 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa386fb1740 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x56537399acd0 0x5653739999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa386fb1740 4 client.15426 mds_command: new command op to 24388 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa386fb1740 1 -- 172.21.15.49:0/2363862887 --> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] -- command(tid 0: {"prefix": "status"}) -- 0x565373880b00 con 0x56537399acd0 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa382086640 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x56537399acd0 0x5653739999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa382086640 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x56537399acd0 0x5653739999e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=827927f0b677ee65 server_cookie=6daba325081ad06f in_seq=0 out_seq=0 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa365ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.171:6832/2067087026 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa365ffb640 1 -- 172.21.15.49:0/2363862887 <== mds.4 v2:172.21.15.171:6832/2067087026 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x565373880b00 con 0x56537399acd0 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.202+0000 7fa365ffb640 10 client.15426 handle_command_reply: tid=0 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 client.15426 shutdown 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 -- 172.21.15.49:0/2363862887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391a220 msgr2=0x5653739268d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391a220 0x5653739268d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa36c01c040 tx=0x7fa36c009930 comp rx=0 tx=0).stop 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 -- 172.21.15.49:0/2363862887 shutdown_connections 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.171:6832/2067087026,v1:172.21.15.171:6833/2067087026] conn(0x56537399acd0 0x5653739999e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x56537391abe0 0x5653739298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56537391a220 0x5653739268d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa365ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.171:6832/2067087026 2024-03-26T18:01:57.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 --2- 172.21.15.49:0/2363862887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565373919050 0x565373926390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 -- 172.21.15.49:0/2363862887 >> 172.21.15.49:0/2363862887 conn(0x565373910030 msgr2=0x56537391dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 -- 172.21.15.49:0/2363862887 shutdown_connections 2024-03-26T18:01:57.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 1 -- 172.21.15.49:0/2363862887 wait complete. 2024-03-26T18:01:57.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa386fb1740 20 client.15426 trim_cache size 0 max 16384 2024-03-26T18:01:57.192 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:01:57.192 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T18:01:57.192 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 4, 2024-03-26T18:01:57.192 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24388, 2024-03-26T18:01:57.192 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T18:01:57.193 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T18:01:57.193 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T18:01:57.193 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 108.75786562099999, 2024-03-26T18:01:57.193 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2024-03-26T18:01:57.193 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 94, 2024-03-26T18:01:57.193 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T18:01:57.193 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 165.349229603 2024-03-26T18:01:57.193 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:01:57.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 -- 172.21.15.49:0/4000704534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa3680a2f20 msgr2=0x7fa3680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa3680a2f20 0x7fa3680a53e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x56537391a120 tx=0x7fa374009500 comp rx=0 tx=0).stop 2024-03-26T18:01:57.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 -- 172.21.15.49:0/4000704534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa37c102360 msgr2=0x7fa37c197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa37c102360 0x7fa37c197c50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa378009230 tx=0x7fa37802c650 comp rx=0 tx=0).stop 2024-03-26T18:01:57.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 -- 172.21.15.49:0/4000704534 shutdown_connections 2024-03-26T18:01:57.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7fa3680a2f20 0x7fa3680a53e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa37c104c90 0x7fa37c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa37c1075c0 0x7fa37c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 --2- 172.21.15.49:0/4000704534 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa37c102360 0x7fa37c197c50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.194 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 -- 172.21.15.49:0/4000704534 >> 172.21.15.49:0/4000704534 conn(0x7fa37c0fbd70 msgr2=0x7fa37c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 -- 172.21.15.49:0/4000704534 shutdown_connections 2024-03-26T18:01:57.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.203+0000 7fa3842ea640 1 -- 172.21.15.49:0/4000704534 wait complete. 2024-03-26T18:01:57.196 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-26T18:01:57.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.324+0000 7f5829844640 1 -- 172.21.15.49:0/3914272692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5824107660 msgr2=0x7f5824109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.324+0000 7f5829844640 1 --2- 172.21.15.49:0/3914272692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5824107660 0x7f5824109a50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f581c009be0 tx=0x7f581c02bef0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.324+0000 7f5829844640 1 -- 172.21.15.49:0/3914272692 shutdown_connections 2024-03-26T18:01:57.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.324+0000 7f5829844640 1 --2- 172.21.15.49:0/3914272692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5824107660 0x7f5824109a50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.324+0000 7f5829844640 1 --2- 172.21.15.49:0/3914272692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5824104d30 0x7f5824107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.324+0000 7f5829844640 1 --2- 172.21.15.49:0/3914272692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5824102400 0x7f58241047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.324+0000 7f5829844640 1 -- 172.21.15.49:0/3914272692 >> 172.21.15.49:0/3914272692 conn(0x7f58240fbd70 msgr2=0x7f58240fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f5829844640 1 -- 172.21.15.49:0/3914272692 shutdown_connections 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f5829844640 1 -- 172.21.15.49:0/3914272692 wait complete. 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f5829844640 1 Processor -- start 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f5829844640 1 -- start start 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f5829844640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5824102400 0x7f5824197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f5829844640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5824104d30 0x7f58241980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f5829844640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5824107660 0x7f582419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f5829844640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f582410d560 con 0x7f5824107660 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f5829844640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f582410d3e0 con 0x7f5824104d30 2024-03-26T18:01:57.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f5829844640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f582410d6e0 con 0x7f5824102400 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.325+0000 7f58237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5824107660 0x7f582419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f581bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5824104d30 0x7f58241980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f5822ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5824102400 0x7f5824197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f58237fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5824107660 0x7f582419f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33368/0 (socket says 172.21.15.49:33368) 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f58237fe640 1 -- 172.21.15.49:0/614656033 learned_addr learned my addr 172.21.15.49:0/614656033 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f581bfff640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5824104d30 0x7f58241980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48628/0 (socket says 172.21.15.49:48628) 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f58237fe640 1 -- 172.21.15.49:0/614656033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5824102400 msgr2=0x7f5824197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f58237fe640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5824102400 0x7f5824197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f58237fe640 1 -- 172.21.15.49:0/614656033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5824104d30 msgr2=0x7f58241980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f58237fe640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5824104d30 0x7f58241980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f58237fe640 1 -- 172.21.15.49:0/614656033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582419f7d0 con 0x7f5824107660 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f581bfff640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5824104d30 0x7f58241980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f58237fe640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5824107660 0x7f582419f160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f581c036040 tx=0x7f581c0337b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f5828842640 1 -- 172.21.15.49:0/614656033 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c043020 con 0x7f5824107660 2024-03-26T18:01:57.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f5828842640 1 -- 172.21.15.49:0/614656033 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f581c02ca60 con 0x7f5824107660 2024-03-26T18:01:57.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f5829844640 1 -- 172.21.15.49:0/614656033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582419fa60 con 0x7f5824107660 2024-03-26T18:01:57.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f5828842640 1 -- 172.21.15.49:0/614656033 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c034690 con 0x7f5824107660 2024-03-26T18:01:57.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.326+0000 7f5829844640 1 -- 172.21.15.49:0/614656033 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582419ffb0 con 0x7f5824107660 2024-03-26T18:01:57.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.328+0000 7f5828842640 1 -- 172.21.15.49:0/614656033 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f581c046020 con 0x7f5824107660 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f582c42f740 1 Processor -- start 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f582c42f740 1 -- start start 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f582c42f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a71060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f582c42f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a72290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f582c42f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a727d0 0x555582a79060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f582c42f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5555829d5270 con 0x555582a70c40 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f582c42f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x55558297dc80 con 0x555582a71e10 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f582c42f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5555829d1220 con 0x555582a727d0 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f5828842640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f58100a2f20 0x7f58100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f581bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a71060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f581bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a71060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33380/0 (socket says 172.21.15.49:33380) 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f5822ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a72290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f581bfff640 1 -- 172.21.15.49:0/3166298677 learned_addr learned my addr 172.21.15.49:0/3166298677 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f581bfff640 1 --2- 172.21.15.49:0/3166298677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a727d0 0x555582a79060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f58237fe640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f58100a2f20 0x7f58100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f5828842640 1 -- 172.21.15.49:0/614656033 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f581c0f1d60 con 0x7f5824107660 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f5822ffd640 1 -- 172.21.15.49:0/3166298677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a727d0 msgr2=0x555582a79060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f5822ffd640 1 --2- 172.21.15.49:0/3166298677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a727d0 0x555582a79060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f5822ffd640 1 -- 172.21.15.49:0/3166298677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 msgr2=0x555582a71060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f5822ffd640 1 --2- 172.21.15.49:0/3166298677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a71060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.329+0000 7f5822ffd640 1 -- 172.21.15.49:0/3166298677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558298ffb0 con 0x555582a71e10 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f5822ffd640 1 --2- 172.21.15.49:0/3166298677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a72290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5814009c20 tx=0x7f5814011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72a583f656610699 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f58237fe640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f58100a2f20 0x7f58100a53e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f581c0be0a0 tx=0x7f581c0b35f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f5819ffb640 1 -- 172.21.15.49:0/3166298677 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581401e070 con 0x555582a71e10 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f5819ffb640 1 -- 172.21.15.49:0/3166298677 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5814019910 con 0x555582a71e10 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f5819ffb640 1 -- 172.21.15.49:0/3166298677 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581400ea80 con 0x555582a71e10 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f582c42f740 1 -- 172.21.15.49:0/3166298677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 msgr2=0x555582a72290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f582c42f740 1 --2- 172.21.15.49:0/3166298677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a72290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5814009c20 tx=0x7f5814011810 comp rx=0 tx=0).stop 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f582c42f740 1 -- 172.21.15.49:0/3166298677 shutdown_connections 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f582c42f740 1 --2- 172.21.15.49:0/3166298677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a727d0 0x555582a79060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f582c42f740 1 --2- 172.21.15.49:0/3166298677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a72290 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f582c42f740 1 --2- 172.21.15.49:0/3166298677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a71060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f582c42f740 1 -- 172.21.15.49:0/3166298677 >> 172.21.15.49:0/3166298677 conn(0x555582a6c030 msgr2=0x555582a6c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f582c42f740 1 -- 172.21.15.49:0/3166298677 shutdown_connections 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.330+0000 7f582c42f740 1 -- 172.21.15.49:0/3166298677 wait complete. 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f582c42f740 1 -- start start 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f582c42f740 1 Processor -- start 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f582c42f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a70c40 0x555582a7b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f582c42f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a7b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f582c42f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a82700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f582c42f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555582a797e0 con 0x555582a727d0 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f582c42f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555582990fc0 con 0x555582a71e10 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f582c42f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555582a79960 con 0x555582a70c40 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f582c42f740 10 client.0 _ll_register_callbacks cb 0x55558298b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f581bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a82700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f581bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a82700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33396/0 (socket says 172.21.15.49:33396) 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f581bfff640 1 -- 172.21.15.49:0/3759358800 learned_addr learned my addr 172.21.15.49:0/3759358800 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f581bfff640 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a70c40 0x555582a7b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.331+0000 7f5822ffd640 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a7b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f581bfff640 1 -- 172.21.15.49:0/3759358800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a70c40 msgr2=0x555582a7b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f581bfff640 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a70c40 0x555582a7b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f581bfff640 1 -- 172.21.15.49:0/3759358800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 msgr2=0x555582a7b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f581bfff640 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a7b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f581bfff640 1 -- 172.21.15.49:0/3759358800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555826e92d0 con 0x555582a727d0 2024-03-26T18:01:57.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f5822ffd640 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a7b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f581bfff640 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a82700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f580c013f70 tx=0x7f580c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d08afed36a66ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f5802ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f5802ffd640 1 -- 172.21.15.49:0/3759358800 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c004030 con 0x555582a727d0 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f5802ffd640 1 -- 172.21.15.49:0/3759358800 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f580c0041d0 con 0x555582a727d0 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f582c42f740 1 -- 172.21.15.49:0/3759358800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5814019910 con 0x555582a727d0 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f5802ffd640 1 -- 172.21.15.49:0/3759358800 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f580c0043d0 con 0x555582a727d0 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f582c42f740 10 client.15438 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f582c42f740 1 -- 172.21.15.49:0/3759358800 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555827b36b0 con 0x555582a727d0 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.332+0000 7f5802ffd640 1 -- 172.21.15.49:0/3759358800 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5814019910 con 0x555582a727d0 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.333+0000 7f5802ffd640 1 -- 172.21.15.49:0/3759358800 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f580c004b40 con 0x555582a727d0 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.333+0000 7f582c42f740 10 client.15438 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.333+0000 7f582c42f740 10 client.15438 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.333+0000 7f582c42f740 20 client.15438 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:57.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.333+0000 7f582c42f740 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x555582a88050 0x555582a894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.333+0000 7f582c42f740 4 client.15438 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.333+0000 7f582c42f740 1 -- 172.21.15.49:0/3759358800 --> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555829e27b0 con 0x555582a88050 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.333+0000 7f5822ffd640 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x555582a88050 0x555582a894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.333+0000 7f5822ffd640 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x555582a88050 0x555582a894c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b097d07abb93b53d server_cookie=8e6663368e5feedd in_seq=0 out_seq=0 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.333+0000 7f5802ffd640 10 client.15438 ms_handle_connect on v2:172.21.15.115:6832/3604276420 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f5802ffd640 1 -- 172.21.15.49:0/3759358800 <== mds.3 v2:172.21.15.115:6832/3604276420 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5555829e27b0 con 0x555582a88050 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f5802ffd640 10 client.15438 handle_command_reply: tid=0 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 client.15438 shutdown 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 -- 172.21.15.49:0/3759358800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 msgr2=0x555582a82700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a82700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f580c013f70 tx=0x7f580c009260 comp rx=0 tx=0).stop 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 -- 172.21.15.49:0/3759358800 shutdown_connections 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x555582a88050 0x555582a894c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a82700 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a7b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f5802ffd640 0 client.15438 ms_handle_reset on v2:172.21.15.115:6832/3604276420 2024-03-26T18:01:57.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 --2- 172.21.15.49:0/3759358800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a70c40 0x555582a7b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 -- 172.21.15.49:0/3759358800 >> 172.21.15.49:0/3759358800 conn(0x555582a6c030 msgr2=0x555582a6b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 -- 172.21.15.49:0/3759358800 shutdown_connections 2024-03-26T18:01:57.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 1 -- 172.21.15.49:0/3759358800 wait complete. 2024-03-26T18:01:57.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.334+0000 7f582c42f740 20 client.15438 trim_cache size 0 max 16384 2024-03-26T18:01:57.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.352+0000 7f582c42f740 1 Processor -- start 2024-03-26T18:01:57.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f582c42f740 1 -- start start 2024-03-26T18:01:57.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f582c42f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a7ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f582c42f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a71e10 0x555582a7b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f582c42f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a727d0 0x555582a79d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f582c42f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555582a87800 con 0x555582a70c40 2024-03-26T18:01:57.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f582c42f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555582a87ed0 con 0x555582a727d0 2024-03-26T18:01:57.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f582c42f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555582a885a0 con 0x555582a71e10 2024-03-26T18:01:57.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f581bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a7ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f581bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a7ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33406/0 (socket says 172.21.15.49:33406) 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f581bfff640 1 -- 172.21.15.49:0/2739954854 learned_addr learned my addr 172.21.15.49:0/2739954854 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f5822ffd640 1 --2- 172.21.15.49:0/2739954854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a71e10 0x555582a7b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f581bfff640 1 --2- 172.21.15.49:0/2739954854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a727d0 0x555582a79d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f5822ffd640 1 -- 172.21.15.49:0/2739954854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a727d0 msgr2=0x555582a79d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f5822ffd640 1 --2- 172.21.15.49:0/2739954854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a727d0 0x555582a79d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f5822ffd640 1 -- 172.21.15.49:0/2739954854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 msgr2=0x555582a7ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f5822ffd640 1 --2- 172.21.15.49:0/2739954854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a7ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f5822ffd640 1 -- 172.21.15.49:0/2739954854 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555582a71890 con 0x555582a71e10 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f581bfff640 1 --2- 172.21.15.49:0/2739954854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a7ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.353+0000 7f5822ffd640 1 --2- 172.21.15.49:0/2739954854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a71e10 0x555582a7b340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5814011d10 tx=0x7f5814005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dc8466117e9a37b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f58197fa640 1 -- 172.21.15.49:0/2739954854 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581401e040 con 0x555582a71e10 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f58197fa640 1 -- 172.21.15.49:0/2739954854 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f581401f040 con 0x555582a71e10 2024-03-26T18:01:57.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f58197fa640 1 -- 172.21.15.49:0/2739954854 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814003850 con 0x555582a71e10 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f582c42f740 1 -- 172.21.15.49:0/2739954854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a71e10 msgr2=0x555582a7b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f582c42f740 1 --2- 172.21.15.49:0/2739954854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a71e10 0x555582a7b340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5814011d10 tx=0x7f5814005870 comp rx=0 tx=0).stop 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f582c42f740 1 -- 172.21.15.49:0/2739954854 shutdown_connections 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f582c42f740 1 --2- 172.21.15.49:0/2739954854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a71e10 0x555582a7b340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f582c42f740 1 --2- 172.21.15.49:0/2739954854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a727d0 0x555582a79d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f582c42f740 1 --2- 172.21.15.49:0/2739954854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a70c40 0x555582a7ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f582c42f740 1 -- 172.21.15.49:0/2739954854 >> 172.21.15.49:0/2739954854 conn(0x555582a6c030 msgr2=0x555582a74e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f582c42f740 1 -- 172.21.15.49:0/2739954854 shutdown_connections 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f582c42f740 1 -- 172.21.15.49:0/2739954854 wait complete. 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.354+0000 7f582c42f740 1 -- start start 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f582c42f740 1 Processor -- start 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f582c42f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a70c40 0x555582a74b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f582c42f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a75040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f582c42f740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a73110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f582c42f740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x555582a89340 con 0x555582a727d0 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f582c42f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x555582a88c70 con 0x555582a71e10 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f582c42f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555582a89a10 con 0x555582a70c40 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f582c42f740 10 client.0 _ll_register_callbacks cb 0x7f580c0139b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f581bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f5822ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a75040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f581bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a73110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33418/0 (socket says 172.21.15.49:33418) 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f581bfff640 1 -- 172.21.15.49:0/454563556 learned_addr learned my addr 172.21.15.49:0/454563556 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f581bfff640 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a70c40 0x555582a74b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f5822ffd640 1 -- 172.21.15.49:0/454563556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a70c40 msgr2=0x555582a74b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f5822ffd640 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a70c40 0x555582a74b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f5822ffd640 1 -- 172.21.15.49:0/454563556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 msgr2=0x555582a73110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f5822ffd640 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a73110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f5822ffd640 1 -- 172.21.15.49:0/454563556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555582a73940 con 0x555582a71e10 2024-03-26T18:01:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.355+0000 7f581bfff640 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a73110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f5822ffd640 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a75040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x555582a70920 tx=0x7f5814005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad458dcbadacde3f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f5802ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-03-26T18:01:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f5802ffd640 1 -- 172.21.15.49:0/454563556 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581401e040 con 0x555582a71e10 2024-03-26T18:01:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f5802ffd640 1 -- 172.21.15.49:0/454563556 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f58140050c0 con 0x555582a71e10 2024-03-26T18:01:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f582c42f740 1 -- 172.21.15.49:0/454563556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f581401f040 con 0x555582a71e10 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f5802ffd640 1 -- 172.21.15.49:0/454563556 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5814009430 con 0x555582a71e10 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f582c42f740 10 client.25340 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f582c42f740 1 -- 172.21.15.49:0/454563556 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555582af2900 con 0x555582a71e10 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f5802ffd640 1 -- 172.21.15.49:0/454563556 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f581401f040 con 0x555582a71e10 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f5802ffd640 1 -- 172.21.15.49:0/454563556 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5814003b10 con 0x555582a71e10 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f582c42f740 10 client.25340 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f582c42f740 10 client.25340 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f582c42f740 20 client.25340 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f582c42f740 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x555582af6dc0 0x555582af5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f582c42f740 4 client.25340 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f582c42f740 1 -- 172.21.15.49:0/454563556 --> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] -- command(tid 0: {"prefix": "status"}) -- 0x5555829dcb00 con 0x555582af6dc0 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.356+0000 7f581bfff640 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x555582af6dc0 0x555582af5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f5802ffd640 10 client.25340 ms_handle_connect on v2:172.21.15.115:6832/3604276420 2024-03-26T18:01:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f581bfff640 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x555582af6dc0 0x555582af5ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d8d75e90e3c08f8 server_cookie=9984353e8778c894 in_seq=0 out_seq=0 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f5802ffd640 1 -- 172.21.15.49:0/454563556 <== mds.3 v2:172.21.15.115:6832/3604276420 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5555829dcb00 con 0x555582af6dc0 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f5802ffd640 10 client.25340 handle_command_reply: tid=0 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 client.25340 shutdown 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 -- 172.21.15.49:0/454563556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 msgr2=0x555582a75040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a75040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x555582a70920 tx=0x7f5814005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 -- 172.21.15.49:0/454563556 shutdown_connections 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.115:6832/3604276420,v1:172.21.15.115:6833/3604276420] conn(0x555582af6dc0 0x555582af5ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555582a727d0 0x555582a73110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555582a71e10 0x555582a75040 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 --2- 172.21.15.49:0/454563556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555582a70c40 0x555582a74b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f5802ffd640 0 client.25340 ms_handle_reset on v2:172.21.15.115:6832/3604276420 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 -- 172.21.15.49:0/454563556 >> 172.21.15.49:0/454563556 conn(0x555582a6c030 msgr2=0x555582a6f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 -- 172.21.15.49:0/454563556 shutdown_connections 2024-03-26T18:01:57.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 1 -- 172.21.15.49:0/454563556 wait complete. 2024-03-26T18:01:57.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.357+0000 7f582c42f740 20 client.25340 trim_cache size 0 max 16384 2024-03-26T18:01:57.347 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:01:57.347 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T18:01:57.347 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 3, 2024-03-26T18:01:57.347 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24428, 2024-03-26T18:01:57.347 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T18:01:57.347 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T18:01:57.347 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T18:01:57.347 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 109.917549897, 2024-03-26T18:01:57.347 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2024-03-26T18:01:57.348 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 94, 2024-03-26T18:01:57.348 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T18:01:57.348 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 162.68840468299999 2024-03-26T18:01:57.348 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:01:57.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 -- 172.21.15.49:0/614656033 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f58100a2f20 msgr2=0x7f58100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f58100a2f20 0x7f58100a53e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f581c0be0a0 tx=0x7f581c0b35f0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 -- 172.21.15.49:0/614656033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5824107660 msgr2=0x7f582419f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5824107660 0x7f582419f160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f581c036040 tx=0x7f581c0337b0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 -- 172.21.15.49:0/614656033 shutdown_connections 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f58100a2f20 0x7f58100a53e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5824107660 0x7f582419f160 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5824104d30 0x7f58241980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 --2- 172.21.15.49:0/614656033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5824102400 0x7f5824197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 -- 172.21.15.49:0/614656033 >> 172.21.15.49:0/614656033 conn(0x7f58240fbd70 msgr2=0x7f5824102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 -- 172.21.15.49:0/614656033 shutdown_connections 2024-03-26T18:01:57.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.358+0000 7f5829844640 1 -- 172.21.15.49:0/614656033 wait complete. 2024-03-26T18:01:57.351 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-26T18:01:57.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.478+0000 7f16589f3640 1 -- 172.21.15.49:0/3433342036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16541075c0 msgr2=0x7f1654109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.478+0000 7f16589f3640 1 --2- 172.21.15.49:0/3433342036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16541075c0 0x7f1654109a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f164c009ca0 tx=0x7f164c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.478+0000 7f16589f3640 1 -- 172.21.15.49:0/3433342036 shutdown_connections 2024-03-26T18:01:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.478+0000 7f16589f3640 1 --2- 172.21.15.49:0/3433342036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16541075c0 0x7f1654109a10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.478+0000 7f16589f3640 1 --2- 172.21.15.49:0/3433342036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1654104c90 0x7f1654107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.478+0000 7f16589f3640 1 --2- 172.21.15.49:0/3433342036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1654102360 0x7f1654104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.478+0000 7f16589f3640 1 -- 172.21.15.49:0/3433342036 >> 172.21.15.49:0/3433342036 conn(0x7f16540fbd70 msgr2=0x7f16540fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.478+0000 7f16589f3640 1 -- 172.21.15.49:0/3433342036 shutdown_connections 2024-03-26T18:01:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.478+0000 7f16589f3640 1 -- 172.21.15.49:0/3433342036 wait complete. 2024-03-26T18:01:57.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f16589f3640 1 Processor -- start 2024-03-26T18:01:57.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f16589f3640 1 -- start start 2024-03-26T18:01:57.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f16589f3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1654102360 0x7f1654197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f16589f3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1654104c90 0x7f16541980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f16589f3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16541075c0 0x7f165419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f16589f3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f165410d580 con 0x7f1654104c90 2024-03-26T18:01:57.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f16589f3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f165410d400 con 0x7f1654102360 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f16589f3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f165410d700 con 0x7f16541075c0 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f1649d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1654104c90 0x7f16541980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f165259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1654102360 0x7f1654197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f1649d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1654104c90 0x7f16541980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33438/0 (socket says 172.21.15.49:33438) 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f1649d9b640 1 -- 172.21.15.49:0/2160095450 learned_addr learned my addr 172.21.15.49:0/2160095450 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f165259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1654102360 0x7f1654197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48686/0 (socket says 172.21.15.49:48686) 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f1652d9d640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16541075c0 0x7f165419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f1649d9b640 1 -- 172.21.15.49:0/2160095450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16541075c0 msgr2=0x7f165419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f1649d9b640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16541075c0 0x7f165419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f1649d9b640 1 -- 172.21.15.49:0/2160095450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1654102360 msgr2=0x7f1654197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.479+0000 7f1649d9b640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1654102360 0x7f1654197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.480+0000 7f1649d9b640 1 -- 172.21.15.49:0/2160095450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165419f7b0 con 0x7f1654104c90 2024-03-26T18:01:57.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.480+0000 7f165259c640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1654102360 0x7f1654197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.480+0000 7f1649d9b640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1654104c90 0x7f16541980d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f163c00e9c0 tx=0x7f163c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.480+0000 7f164bfff640 1 -- 172.21.15.49:0/2160095450 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c00cd80 con 0x7f1654104c90 2024-03-26T18:01:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.480+0000 7f16589f3640 1 -- 172.21.15.49:0/2160095450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165419faa0 con 0x7f1654104c90 2024-03-26T18:01:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.480+0000 7f164bfff640 1 -- 172.21.15.49:0/2160095450 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163c005500 con 0x7f1654104c90 2024-03-26T18:01:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.480+0000 7f164bfff640 1 -- 172.21.15.49:0/2160095450 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c010690 con 0x7f1654104c90 2024-03-26T18:01:57.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.480+0000 7f16589f3640 1 -- 172.21.15.49:0/2160095450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f165419fff0 con 0x7f1654104c90 2024-03-26T18:01:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.482+0000 7f164bfff640 1 -- 172.21.15.49:0/2160095450 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f163c0046c0 con 0x7f1654104c90 2024-03-26T18:01:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.482+0000 7f165b5dc740 1 Processor -- start 2024-03-26T18:01:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165b5dc740 1 -- start start 2024-03-26T18:01:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f164bfff640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f16400a2f20 0x7f16400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f42c40 0x557306f43060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 0x557306f44290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 0x557306f4b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165b5dc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557306ea7270 con 0x557306f42c40 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165b5dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557306e4fc80 con 0x557306f447d0 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165b5dc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557306ea3220 con 0x557306f43e10 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f1652d9d640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f16400a2f20 0x7f16400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f42c40 0x557306f43060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f1649d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 0x557306f44290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f42c40 0x557306f43060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33450/0 (socket says 172.21.15.49:33450) 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165259c640 1 -- 172.21.15.49:0/4285693393 learned_addr learned my addr 172.21.15.49:0/4285693393 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f164bfff640 1 -- 172.21.15.49:0/2160095450 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f163c0cfab0 con 0x7f1654104c90 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f1652d9d640 1 --2- 172.21.15.49:0/4285693393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 0x557306f4b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165259c640 1 -- 172.21.15.49:0/4285693393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 msgr2=0x557306f44290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165259c640 1 --2- 172.21.15.49:0/4285693393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 0x557306f44290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165259c640 1 -- 172.21.15.49:0/4285693393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 msgr2=0x557306f4b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165259c640 1 --2- 172.21.15.49:0/4285693393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 0x557306f4b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f165259c640 1 -- 172.21.15.49:0/4285693393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557306e61fb0 con 0x557306f42c40 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f1652d9d640 1 --2- 172.21.15.49:0/4285693393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 0x557306f4b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.483+0000 7f1649d9b640 1 --2- 172.21.15.49:0/4285693393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 0x557306f44290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.484+0000 7f1652d9d640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f16400a2f20 0x7f16400a53e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f164c02c630 tx=0x7f164c032360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.484+0000 7f165259c640 1 --2- 172.21.15.49:0/4285693393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f42c40 0x557306f43060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f16440115c0 tx=0x7f1644011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea0a0438c738dea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.484+0000 7f1648d99640 1 -- 172.21.15.49:0/4285693393 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164400f640 con 0x557306f42c40 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.484+0000 7f1648d99640 1 -- 172.21.15.49:0/4285693393 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f164400fcc0 con 0x557306f42c40 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.484+0000 7f1648d99640 1 -- 172.21.15.49:0/4285693393 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644019d40 con 0x557306f42c40 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.484+0000 7f165b5dc740 1 -- 172.21.15.49:0/4285693393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f42c40 msgr2=0x557306f43060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.484+0000 7f165b5dc740 1 --2- 172.21.15.49:0/4285693393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f42c40 0x557306f43060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f16440115c0 tx=0x7f1644011a80 comp rx=0 tx=0).stop 2024-03-26T18:01:57.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.484+0000 7f165b5dc740 1 -- 172.21.15.49:0/4285693393 shutdown_connections 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 --2- 172.21.15.49:0/4285693393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 0x557306f44290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 --2- 172.21.15.49:0/4285693393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 0x557306f4b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 --2- 172.21.15.49:0/4285693393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f42c40 0x557306f43060 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 -- 172.21.15.49:0/4285693393 >> 172.21.15.49:0/4285693393 conn(0x557306f3e030 msgr2=0x557306f3e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 -- 172.21.15.49:0/4285693393 shutdown_connections 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 -- 172.21.15.49:0/4285693393 wait complete. 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 -- start start 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 Processor -- start 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f42c40 0x557306f4d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f4d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 0x557306f54740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557306f4b7e0 con 0x557306f43e10 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557306e62fc0 con 0x557306f447d0 2024-03-26T18:01:57.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557306f4b960 con 0x557306f42c40 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165b5dc740 10 client.0 _ll_register_callbacks cb 0x557306e5d770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f1649d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f4d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f1649d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f4d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33466/0 (socket says 172.21.15.49:33466) 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f1649d9b640 1 -- 172.21.15.49:0/3726381525 learned_addr learned my addr 172.21.15.49:0/3726381525 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f1652d9d640 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 0x557306f54740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.485+0000 7f165259c640 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f42c40 0x557306f4d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1649d9b640 1 -- 172.21.15.49:0/3726381525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f42c40 msgr2=0x557306f4d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1649d9b640 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f42c40 0x557306f4d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1649d9b640 1 -- 172.21.15.49:0/3726381525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 msgr2=0x557306f54740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1649d9b640 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 0x557306f54740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1649d9b640 1 -- 172.21.15.49:0/3726381525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557306bbb2d0 con 0x557306f43e10 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1649d9b640 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f4d700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f163c098830 tx=0x7f163c0097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afc6774f5e72427 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:57.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1639ffb640 1 -- 172.21.15.49:0/3726381525 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c0b5190 con 0x557306f43e10 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1639ffb640 1 -- 172.21.15.49:0/3726381525 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f163c09c0c0 con 0x557306f43e10 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f165b5dc740 1 -- 172.21.15.49:0/3726381525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f164400fcc0 con 0x557306f43e10 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1639ffb640 1 -- 172.21.15.49:0/3726381525 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f163c01a0c0 con 0x557306f43e10 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f165b5dc740 10 client.15456 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f165b5dc740 1 -- 172.21.15.49:0/3726381525 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557306c856b0 con 0x557306f43e10 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.486+0000 7f1639ffb640 1 -- 172.21.15.49:0/3726381525 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f164400fcc0 con 0x557306f43e10 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.487+0000 7f1639ffb640 1 -- 172.21.15.49:0/3726381525 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f163c0a43e0 con 0x557306f43e10 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.487+0000 7f165b5dc740 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.487+0000 7f165b5dc740 10 client.15456 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.487+0000 7f165b5dc740 20 client.15456 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.487+0000 7f165b5dc740 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x557306f5a070 0x557306f5b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.487+0000 7f165b5dc740 4 client.15456 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.487+0000 7f165b5dc740 1 -- 172.21.15.49:0/3726381525 --> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557306eb47b0 con 0x557306f5a070 2024-03-26T18:01:57.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.487+0000 7f165259c640 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x557306f5a070 0x557306f5b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.487+0000 7f165259c640 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x557306f5a070 0x557306f5b4e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac62bc8f5b0aadff server_cookie=d4a8a321443cb6cf in_seq=0 out_seq=0 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.487+0000 7f1639ffb640 10 client.15456 ms_handle_connect on v2:172.21.15.171:6838/1082475840 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.488+0000 7f1639ffb640 1 -- 172.21.15.49:0/3726381525 <== mds.2 v2:172.21.15.171:6838/1082475840 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557306eb47b0 con 0x557306f5a070 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.488+0000 7f1639ffb640 10 client.15456 handle_command_reply: tid=0 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.488+0000 7f165b5dc740 1 client.15456 shutdown 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.488+0000 7f165b5dc740 1 -- 172.21.15.49:0/3726381525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 msgr2=0x557306f4d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.488+0000 7f165b5dc740 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f4d700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f163c098830 tx=0x7f163c0097f0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.488+0000 7f165b5dc740 1 -- 172.21.15.49:0/3726381525 shutdown_connections 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.488+0000 7f165b5dc740 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x557306f5a070 0x557306f5b4e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.489+0000 7f165b5dc740 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f447d0 0x557306f54740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.489+0000 7f165b5dc740 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f4d700 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.489+0000 7f165b5dc740 1 --2- 172.21.15.49:0/3726381525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f42c40 0x557306f4d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.489+0000 7f1639ffb640 0 client.15456 ms_handle_reset on v2:172.21.15.171:6838/1082475840 2024-03-26T18:01:57.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.489+0000 7f165b5dc740 1 -- 172.21.15.49:0/3726381525 >> 172.21.15.49:0/3726381525 conn(0x557306f3e030 msgr2=0x557306f3d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.489+0000 7f165b5dc740 1 -- 172.21.15.49:0/3726381525 shutdown_connections 2024-03-26T18:01:57.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.489+0000 7f165b5dc740 1 -- 172.21.15.49:0/3726381525 wait complete. 2024-03-26T18:01:57.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.489+0000 7f165b5dc740 20 client.15456 trim_cache size 0 max 16384 2024-03-26T18:01:57.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f165b5dc740 1 Processor -- start 2024-03-26T18:01:57.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f165b5dc740 1 -- start start 2024-03-26T18:01:57.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 0x557306f4bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f54700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f447d0 0x557306f3c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f165b5dc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557306f59820 con 0x557306f43e10 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f165b5dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557306f59ef0 con 0x557306f42c40 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f165b5dc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557306f5a5c0 con 0x557306f447d0 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f1649d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f54700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f1649d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f54700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33470/0 (socket says 172.21.15.49:33470) 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f1649d9b640 1 -- 172.21.15.49:0/764199883 learned_addr learned my addr 172.21.15.49:0/764199883 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f165259c640 1 --2- 172.21.15.49:0/764199883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 0x557306f4bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f1652d9d640 1 --2- 172.21.15.49:0/764199883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f447d0 0x557306f3c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f1649d9b640 1 -- 172.21.15.49:0/764199883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f447d0 msgr2=0x557306f3c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f1649d9b640 1 --2- 172.21.15.49:0/764199883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f447d0 0x557306f3c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f1649d9b640 1 -- 172.21.15.49:0/764199883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 msgr2=0x557306f4bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f1649d9b640 1 --2- 172.21.15.49:0/764199883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 0x557306f4bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f1649d9b640 1 -- 172.21.15.49:0/764199883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557306f43890 con 0x557306f43e10 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.507+0000 7f1652d9d640 1 --2- 172.21.15.49:0/764199883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f447d0 0x557306f3c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f1649d9b640 1 --2- 172.21.15.49:0/764199883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f54700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f163c005760 tx=0x7f163c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7f2740d7d964c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f163bfff640 1 -- 172.21.15.49:0/764199883 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c09b7c0 con 0x557306f43e10 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f163bfff640 1 -- 172.21.15.49:0/764199883 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163c0b5190 con 0x557306f43e10 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f163bfff640 1 -- 172.21.15.49:0/764199883 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c09b2b0 con 0x557306f43e10 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f165b5dc740 1 -- 172.21.15.49:0/764199883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 msgr2=0x557306f54700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f165b5dc740 1 --2- 172.21.15.49:0/764199883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f54700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f163c005760 tx=0x7f163c009700 comp rx=0 tx=0).stop 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f165b5dc740 1 -- 172.21.15.49:0/764199883 shutdown_connections 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f165b5dc740 1 --2- 172.21.15.49:0/764199883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f447d0 0x557306f3c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f165b5dc740 1 --2- 172.21.15.49:0/764199883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f43e10 0x557306f54700 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f165b5dc740 1 --2- 172.21.15.49:0/764199883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 0x557306f4bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f165b5dc740 1 -- 172.21.15.49:0/764199883 >> 172.21.15.49:0/764199883 conn(0x557306f3e030 msgr2=0x557306f3e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f165b5dc740 1 -- 172.21.15.49:0/764199883 shutdown_connections 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.508+0000 7f165b5dc740 1 -- 172.21.15.49:0/764199883 wait complete. 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165b5dc740 1 -- start start 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165b5dc740 1 Processor -- start 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 0x557306f528a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 0x557306f52de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165b5dc740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f447d0 0x557306f53320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165b5dc740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557306f5b360 con 0x557306f447d0 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165b5dc740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x557306f5ac90 con 0x557306f42c40 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165b5dc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557306f5ba30 con 0x557306f43e10 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165b5dc740 10 client.0 _ll_register_callbacks cb 0x7f163c004a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f1652d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f447d0 0x557306f53320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 0x557306f528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f1652d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f447d0 0x557306f53320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33486/0 (socket says 172.21.15.49:33486) 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f1652d9d640 1 -- 172.21.15.49:0/1084703398 learned_addr learned my addr 172.21.15.49:0/1084703398 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f1649d9b640 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 0x557306f52de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165259c640 1 -- 172.21.15.49:0/1084703398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 msgr2=0x557306f52de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165259c640 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 0x557306f52de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165259c640 1 -- 172.21.15.49:0/1084703398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f447d0 msgr2=0x557306f53320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165259c640 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f447d0 0x557306f53320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f165259c640 1 -- 172.21.15.49:0/1084703398 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557306f53b50 con 0x557306f42c40 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.509+0000 7f1652d9d640 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f447d0 0x557306f53320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f165259c640 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 0x557306f528a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f164401b040 tx=0x7f1644009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace0c639fe6bb949 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f1639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f1639ffb640 1 -- 172.21.15.49:0/1084703398 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644033060 con 0x557306f42c40 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f165b5dc740 1 -- 172.21.15.49:0/1084703398 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f163c0b5190 con 0x557306f42c40 2024-03-26T18:01:57.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f1639ffb640 1 -- 172.21.15.49:0/1084703398 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f1644016040 con 0x557306f42c40 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f1639ffb640 1 -- 172.21.15.49:0/1084703398 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f164403a050 con 0x557306f42c40 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f165b5dc740 10 client.25358 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f165b5dc740 1 -- 172.21.15.49:0/1084703398 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557306fc0030 con 0x557306f42c40 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f1639ffb640 1 -- 172.21.15.49:0/1084703398 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163c0b5190 con 0x557306f42c40 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f1639ffb640 1 -- 172.21.15.49:0/1084703398 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f16440035f0 con 0x557306f42c40 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f165b5dc740 10 client.25358 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f165b5dc740 10 client.25358 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f165b5dc740 20 client.25358 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f165b5dc740 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x557306fc4a00 0x557306fc3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f165b5dc740 4 client.25358 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.510+0000 7f165b5dc740 1 -- 172.21.15.49:0/1084703398 --> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] -- command(tid 0: {"prefix": "status"}) -- 0x557306eaeb00 con 0x557306fc4a00 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f1649d9b640 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x557306fc4a00 0x557306fc3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f1639ffb640 10 client.25358 ms_handle_connect on v2:172.21.15.171:6838/1082475840 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f1649d9b640 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x557306fc4a00 0x557306fc3710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bd5b0c5a53c3281 server_cookie=41c54c42790f5cc9 in_seq=0 out_seq=0 2024-03-26T18:01:57.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f1639ffb640 1 -- 172.21.15.49:0/1084703398 <== mds.2 v2:172.21.15.171:6838/1082475840 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557306eaeb00 con 0x557306fc4a00 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f1639ffb640 10 client.25358 handle_command_reply: tid=0 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 client.25358 shutdown 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 -- 172.21.15.49:0/1084703398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 msgr2=0x557306f528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 0x557306f528a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f164401b040 tx=0x7f1644009c20 comp rx=0 tx=0).stop 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 -- 172.21.15.49:0/1084703398 shutdown_connections 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.171:6838/1082475840,v1:172.21.15.171:6839/1082475840] conn(0x557306fc4a00 0x557306fc3710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557306f43e10 0x557306f52de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f1639ffb640 0 client.25358 ms_handle_reset on v2:172.21.15.171:6838/1082475840 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557306f447d0 0x557306f53320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 --2- 172.21.15.49:0/1084703398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x557306f42c40 0x557306f528a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 -- 172.21.15.49:0/1084703398 >> 172.21.15.49:0/1084703398 conn(0x557306f3e030 msgr2=0x557306f4abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 -- 172.21.15.49:0/1084703398 shutdown_connections 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.511+0000 7f165b5dc740 1 -- 172.21.15.49:0/1084703398 wait complete. 2024-03-26T18:01:57.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f165b5dc740 20 client.25358 trim_cache size 0 max 16384 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24478, 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 111.00193770200001, 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 94, 2024-03-26T18:01:57.501 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T18:01:57.502 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 142.89097092899999 2024-03-26T18:01:57.502 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:01:57.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 -- 172.21.15.49:0/2160095450 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f16400a2f20 msgr2=0x7f16400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f16400a2f20 0x7f16400a53e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f164c02c630 tx=0x7f164c032360 comp rx=0 tx=0).stop 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 -- 172.21.15.49:0/2160095450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1654104c90 msgr2=0x7f16541980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1654104c90 0x7f16541980d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f163c00e9c0 tx=0x7f163c00ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 -- 172.21.15.49:0/2160095450 shutdown_connections 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f16400a2f20 0x7f16400a53e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16541075c0 0x7f165419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1654104c90 0x7f16541980d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 --2- 172.21.15.49:0/2160095450 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1654102360 0x7f1654197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 -- 172.21.15.49:0/2160095450 >> 172.21.15.49:0/2160095450 conn(0x7f16540fbd70 msgr2=0x7f1654102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 -- 172.21.15.49:0/2160095450 shutdown_connections 2024-03-26T18:01:57.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.512+0000 7f16589f3640 1 -- 172.21.15.49:0/2160095450 wait complete. 2024-03-26T18:01:57.505 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-26T18:01:57.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.633+0000 7f5d13352640 1 -- 172.21.15.49:0/2213243938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c104c90 msgr2=0x7f5d0c107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.633+0000 7f5d13352640 1 --2- 172.21.15.49:0/2213243938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c104c90 0x7f5d0c107080 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009900 tx=0x7f5cfc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:57.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.633+0000 7f5d13352640 1 -- 172.21.15.49:0/2213243938 shutdown_connections 2024-03-26T18:01:57.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.633+0000 7f5d13352640 1 --2- 172.21.15.49:0/2213243938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c1075c0 0x7f5d0c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.633+0000 7f5d13352640 1 --2- 172.21.15.49:0/2213243938 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c104c90 0x7f5d0c107080 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.633+0000 7f5d13352640 1 --2- 172.21.15.49:0/2213243938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d0c102360 0x7f5d0c104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.633+0000 7f5d13352640 1 -- 172.21.15.49:0/2213243938 >> 172.21.15.49:0/2213243938 conn(0x7f5d0c0fbd70 msgr2=0x7f5d0c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.633+0000 7f5d13352640 1 -- 172.21.15.49:0/2213243938 shutdown_connections 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.633+0000 7f5d13352640 1 -- 172.21.15.49:0/2213243938 wait complete. 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d13352640 1 Processor -- start 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d13352640 1 -- start start 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d13352640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102360 0x7f5d0c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d13352640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d0c104c90 0x7f5d0c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d13352640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c1075c0 0x7f5d0c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d13352640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c10d570 con 0x7f5d0c1075c0 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d13352640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c10d3f0 con 0x7f5d0c102360 2024-03-26T18:01:57.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d13352640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c10d6f0 con 0x7f5d0c104c90 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d118ef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c1075c0 0x7f5d0c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d118ef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c1075c0 0x7f5d0c19f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33498/0 (socket says 172.21.15.49:33498) 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d118ef640 1 -- 172.21.15.49:0/1196224847 learned_addr learned my addr 172.21.15.49:0/1196224847 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d110ee640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102360 0x7f5d0c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d108ed640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d0c104c90 0x7f5d0c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d118ef640 1 -- 172.21.15.49:0/1196224847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d0c104c90 msgr2=0x7f5d0c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d118ef640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d0c104c90 0x7f5d0c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d118ef640 1 -- 172.21.15.49:0/1196224847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102360 msgr2=0x7f5d0c197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d118ef640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102360 0x7f5d0c197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.634+0000 7f5d118ef640 1 -- 172.21.15.49:0/1196224847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c19f840 con 0x7f5d0c1075c0 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.635+0000 7f5d118ef640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c1075c0 0x7f5d0c19f1a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400eb10 tx=0x7f5d0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.635+0000 7f5d027fc640 1 -- 172.21.15.49:0/1196224847 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04004b20 con 0x7f5d0c1075c0 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.635+0000 7f5d027fc640 1 -- 172.21.15.49:0/1196224847 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d04004cc0 con 0x7f5d0c1075c0 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.635+0000 7f5d13352640 1 -- 172.21.15.49:0/1196224847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c19fb30 con 0x7f5d0c1075c0 2024-03-26T18:01:57.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.635+0000 7f5d027fc640 1 -- 172.21.15.49:0/1196224847 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04010760 con 0x7f5d0c1075c0 2024-03-26T18:01:57.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.635+0000 7f5d13352640 1 -- 172.21.15.49:0/1196224847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c1a0030 con 0x7f5d0c1075c0 2024-03-26T18:01:57.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.637+0000 7f5d027fc640 1 -- 172.21.15.49:0/1196224847 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f5d04005020 con 0x7f5d0c1075c0 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.637+0000 7f5d1600e740 1 Processor -- start 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.637+0000 7f5d1600e740 1 -- start start 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 0x5632fda76060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d027fc640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5cf80a2f20 0x7f5cf80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda76e10 0x5632fda77290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda777d0 0x5632fda7e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d1600e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5632fd9da270 con 0x5632fda777d0 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d1600e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5632fd982c80 con 0x5632fda75c40 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d1600e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5632fd9d6220 con 0x5632fda76e10 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d118ef640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda777d0 0x5632fda7e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d108ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 0x5632fda76060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d110ee640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5cf80a2f20 0x7f5cf80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d108ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 0x5632fda76060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48758/0 (socket says 172.21.15.49:48758) 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d108ed640 1 -- 172.21.15.49:0/477169826 learned_addr learned my addr 172.21.15.49:0/477169826 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d108ed640 1 --2- 172.21.15.49:0/477169826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda76e10 0x5632fda77290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d027fc640 1 -- 172.21.15.49:0/1196224847 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f5d040cfdb0 con 0x7f5d0c1075c0 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d108ed640 1 -- 172.21.15.49:0/477169826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda76e10 msgr2=0x5632fda77290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d108ed640 1 --2- 172.21.15.49:0/477169826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda76e10 0x5632fda77290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d108ed640 1 -- 172.21.15.49:0/477169826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda777d0 msgr2=0x5632fda7e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d108ed640 1 --2- 172.21.15.49:0/477169826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda777d0 0x5632fda7e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d108ed640 1 -- 172.21.15.49:0/477169826 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632fd994fb0 con 0x5632fda75c40 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d118ef640 1 --2- 172.21.15.49:0/477169826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda777d0 0x5632fda7e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d108ed640 1 --2- 172.21.15.49:0/477169826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 0x5632fda76060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c06a9f0 tx=0x7f5cfc02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9317b95fa8eda39 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.638+0000 7f5d110ee640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5cf80a2f20 0x7f5cf80a53e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c199100 tx=0x7f5d08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5cf77fe640 1 -- 172.21.15.49:0/477169826 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc004030 con 0x5632fda75c40 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5cf77fe640 1 -- 172.21.15.49:0/477169826 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc0041d0 con 0x5632fda75c40 2024-03-26T18:01:57.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5cf77fe640 1 -- 172.21.15.49:0/477169826 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc003e30 con 0x5632fda75c40 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5d1600e740 1 -- 172.21.15.49:0/477169826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 msgr2=0x5632fda76060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5d1600e740 1 --2- 172.21.15.49:0/477169826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 0x5632fda76060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c06a9f0 tx=0x7f5cfc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5d1600e740 1 -- 172.21.15.49:0/477169826 shutdown_connections 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5d1600e740 1 --2- 172.21.15.49:0/477169826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda76e10 0x5632fda77290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5d1600e740 1 --2- 172.21.15.49:0/477169826 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda777d0 0x5632fda7e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5d1600e740 1 --2- 172.21.15.49:0/477169826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 0x5632fda76060 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5d1600e740 1 -- 172.21.15.49:0/477169826 >> 172.21.15.49:0/477169826 conn(0x5632fda71030 msgr2=0x5632fda71430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5d1600e740 1 -- 172.21.15.49:0/477169826 shutdown_connections 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.639+0000 7f5d1600e740 1 -- 172.21.15.49:0/477169826 wait complete. 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d1600e740 1 -- start start 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d1600e740 1 Processor -- start 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda75c40 0x5632fda801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda80720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 0x5632fda87760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d1600e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5632fda7e780 con 0x5632fda76e10 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d1600e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5632fd995fc0 con 0x5632fda777d0 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d1600e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5632fda7e900 con 0x5632fda75c40 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d1600e740 10 client.0 _ll_register_callbacks cb 0x5632fd990770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d108ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda80720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d108ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda80720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33508/0 (socket says 172.21.15.49:33508) 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d108ed640 1 -- 172.21.15.49:0/2922451710 learned_addr learned my addr 172.21.15.49:0/2922451710 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d110ee640 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda75c40 0x5632fda801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d118ef640 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 0x5632fda87760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d108ed640 1 -- 172.21.15.49:0/2922451710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda75c40 msgr2=0x5632fda801e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d108ed640 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda75c40 0x5632fda801e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d108ed640 1 -- 172.21.15.49:0/2922451710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 msgr2=0x5632fda87760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d108ed640 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 0x5632fda87760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.640+0000 7f5d108ed640 1 -- 172.21.15.49:0/2922451710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632fd6ee2d0 con 0x5632fda76e10 2024-03-26T18:01:57.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5d108ed640 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda80720 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc000c00 tx=0x7f5cfc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad5812b62d3766 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5cf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/2922451710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc003db0 con 0x5632fda76e10 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5d1600e740 1 -- 172.21.15.49:0/2922451710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cfc0041d0 con 0x5632fda76e10 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/2922451710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f5cfc003f50 con 0x5632fda76e10 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/2922451710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cfc05c050 con 0x5632fda76e10 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5d1600e740 10 client.15474 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5d1600e740 1 -- 172.21.15.49:0/2922451710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632fd7b86b0 con 0x5632fda76e10 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/2922451710 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc0041d0 con 0x5632fda76e10 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/2922451710 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5cfc04c5c0 con 0x5632fda76e10 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5d1600e740 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5d1600e740 10 client.15474 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5d1600e740 20 client.15474 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.641+0000 7f5d1600e740 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5632fda8caa0 0x5632fda8df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.642+0000 7f5d1600e740 4 client.15474 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.642+0000 7f5d1600e740 1 -- 172.21.15.49:0/2922451710 --> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632fd9e77b0 con 0x5632fda8caa0 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.642+0000 7f5d110ee640 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5632fda8caa0 0x5632fda8df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.642+0000 7f5d110ee640 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5632fda8caa0 0x5632fda8df10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=816af7efa5cd8a06 server_cookie=f6c1b016f11c7ae5 in_seq=0 out_seq=0 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.642+0000 7f5cf4ff9640 10 client.15474 ms_handle_connect on v2:172.21.15.115:6836/1993258438 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/2922451710 <== mds.1 v2:172.21.15.115:6836/1993258438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5632fd9e77b0 con 0x5632fda8caa0 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5cf4ff9640 10 client.15474 handle_command_reply: tid=0 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 client.15474 shutdown 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 -- 172.21.15.49:0/2922451710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 msgr2=0x5632fda80720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda80720 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc000c00 tx=0x7f5cfc005b50 comp rx=0 tx=0).stop 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 -- 172.21.15.49:0/2922451710 shutdown_connections 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5632fda8caa0 0x5632fda8df10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 0x5632fda87760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5cf4ff9640 0 client.15474 ms_handle_reset on v2:172.21.15.115:6836/1993258438 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda80720 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 --2- 172.21.15.49:0/2922451710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda75c40 0x5632fda801e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 -- 172.21.15.49:0/2922451710 >> 172.21.15.49:0/2922451710 conn(0x5632fda71030 msgr2=0x5632fda70230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 -- 172.21.15.49:0/2922451710 shutdown_connections 2024-03-26T18:01:57.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 1 -- 172.21.15.49:0/2922451710 wait complete. 2024-03-26T18:01:57.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.643+0000 7f5d1600e740 20 client.15474 trim_cache size 0 max 16384 2024-03-26T18:01:57.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.661+0000 7f5d1600e740 1 Processor -- start 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.661+0000 7f5d1600e740 1 -- start start 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.661+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda75c40 0x5632fda881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.661+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda87720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.661+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 0x5632fda71410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.661+0000 7f5d1600e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5632fda8c250 con 0x5632fda76e10 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.661+0000 7f5d1600e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5632fda8c920 con 0x5632fda777d0 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.661+0000 7f5d1600e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5632fda8cff0 con 0x5632fda75c40 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d108ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda87720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d118ef640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 0x5632fda71410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d110ee640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda75c40 0x5632fda881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d108ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda87720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33510/0 (socket says 172.21.15.49:33510) 2024-03-26T18:01:57.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d108ed640 1 -- 172.21.15.49:0/1336416158 learned_addr learned my addr 172.21.15.49:0/1336416158 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d118ef640 1 -- 172.21.15.49:0/1336416158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda75c40 msgr2=0x5632fda881b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d118ef640 1 --2- 172.21.15.49:0/1336416158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda75c40 0x5632fda881b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d118ef640 1 -- 172.21.15.49:0/1336416158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 msgr2=0x5632fda87720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d118ef640 1 --2- 172.21.15.49:0/1336416158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda87720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d118ef640 1 -- 172.21.15.49:0/1336416158 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632fda709b0 con 0x5632fda777d0 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d108ed640 1 --2- 172.21.15.49:0/1336416158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda87720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5d118ef640 1 --2- 172.21.15.49:0/1336416158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 0x5632fda71410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d0409c090 tx=0x7f5d04005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e639f63235f618 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5cf6ffd640 1 -- 172.21.15.49:0/1336416158 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040a19e0 con 0x5632fda777d0 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5cf6ffd640 1 -- 172.21.15.49:0/1336416158 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0409c800 con 0x5632fda777d0 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.662+0000 7f5cf6ffd640 1 -- 172.21.15.49:0/1336416158 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040a4b10 con 0x5632fda777d0 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 -- 172.21.15.49:0/1336416158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 msgr2=0x5632fda71410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 --2- 172.21.15.49:0/1336416158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 0x5632fda71410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d0409c090 tx=0x7f5d04005b40 comp rx=0 tx=0).stop 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 -- 172.21.15.49:0/1336416158 shutdown_connections 2024-03-26T18:01:57.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 --2- 172.21.15.49:0/1336416158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda777d0 0x5632fda71410 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 --2- 172.21.15.49:0/1336416158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda87720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 --2- 172.21.15.49:0/1336416158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda75c40 0x5632fda881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 -- 172.21.15.49:0/1336416158 >> 172.21.15.49:0/1336416158 conn(0x5632fda71030 msgr2=0x5632fda74b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 -- 172.21.15.49:0/1336416158 shutdown_connections 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 -- 172.21.15.49:0/1336416158 wait complete. 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 -- start start 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.663+0000 7f5d1600e740 1 Processor -- start 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 0x5632fda86ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda873e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d1600e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda777d0 0x5632fda83940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d1600e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5632fda8dd90 con 0x5632fda76e10 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d1600e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5632fda8d6c0 con 0x5632fda75c40 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d1600e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5632fda8e460 con 0x5632fda777d0 2024-03-26T18:01:57.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d1600e740 10 client.0 _ll_register_callbacks cb 0x7f5cfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d108ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda873e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d108ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda873e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33518/0 (socket says 172.21.15.49:33518) 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d110ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 0x5632fda86ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d108ed640 1 -- 172.21.15.49:0/206556172 learned_addr learned my addr 172.21.15.49:0/206556172 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d118ef640 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda777d0 0x5632fda83940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d108ed640 1 -- 172.21.15.49:0/206556172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda777d0 msgr2=0x5632fda83940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d108ed640 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda777d0 0x5632fda83940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d108ed640 1 -- 172.21.15.49:0/206556172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 msgr2=0x5632fda86ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d108ed640 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 0x5632fda86ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d108ed640 1 -- 172.21.15.49:0/206556172 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632fda83f30 con 0x5632fda76e10 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d108ed640 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda873e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc02c7b0 tx=0x7f5cfc04ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2d771dff1b22fa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5cf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:57.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/206556172 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc005d50 con 0x5632fda76e10 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5d1600e740 1 -- 172.21.15.49:0/206556172 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0409c800 con 0x5632fda76e10 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.664+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/206556172 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f5cfc003900 con 0x5632fda76e10 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/206556172 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cfc0057b0 con 0x5632fda76e10 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5d1600e740 10 client.15480 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5d1600e740 1 -- 172.21.15.49:0/206556172 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632fdaf3350 con 0x5632fda76e10 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/206556172 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0409c800 con 0x5632fda76e10 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/206556172 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5cfc04e640 con 0x5632fda76e10 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5d1600e740 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5d1600e740 10 client.15480 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5d1600e740 20 client.15480 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5d1600e740 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5632fdaf7d70 0x5632fdaf6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5d1600e740 4 client.15480 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5d1600e740 1 -- 172.21.15.49:0/206556172 --> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] -- command(tid 0: {"prefix": "status"}) -- 0x5632fd9e1b00 con 0x5632fdaf7d70 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5d110ee640 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5632fdaf7d70 0x5632fdaf6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5cf4ff9640 10 client.15480 ms_handle_connect on v2:172.21.15.115:6836/1993258438 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.665+0000 7f5d110ee640 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5632fdaf7d70 0x5632fdaf6a80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e3f4bd9a21ee8e2 server_cookie=5b54485d0765e629 in_seq=0 out_seq=0 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5cf4ff9640 1 -- 172.21.15.49:0/206556172 <== mds.1 v2:172.21.15.115:6836/1993258438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5632fd9e1b00 con 0x5632fdaf7d70 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5cf4ff9640 10 client.15480 handle_command_reply: tid=0 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 client.15480 shutdown 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 -- 172.21.15.49:0/206556172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 msgr2=0x5632fda873e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda873e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc02c7b0 tx=0x7f5cfc04ce80 comp rx=0 tx=0).stop 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 -- 172.21.15.49:0/206556172 shutdown_connections 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.115:6836/1993258438,v1:172.21.15.115:6837/1993258438] conn(0x5632fdaf7d70 0x5632fdaf6a80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5632fda777d0 0x5632fda83940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5632fda76e10 0x5632fda873e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5cf4ff9640 0 client.15480 ms_handle_reset on v2:172.21.15.115:6836/1993258438 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 --2- 172.21.15.49:0/206556172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5632fda75c40 0x5632fda86ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 -- 172.21.15.49:0/206556172 >> 172.21.15.49:0/206556172 conn(0x5632fda71030 msgr2=0x5632fda74b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 -- 172.21.15.49:0/206556172 shutdown_connections 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 1 -- 172.21.15.49:0/206556172 wait complete. 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.666+0000 7f5d1600e740 20 client.15480 trim_cache size 0 max 16384 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24503, 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T18:01:57.655 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 112.16347043099999, 2024-03-26T18:01:57.656 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2024-03-26T18:01:57.656 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 94, 2024-03-26T18:01:57.656 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T18:01:57.656 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 145.820654803 2024-03-26T18:01:57.656 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 -- 172.21.15.49:0/1196224847 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5cf80a2f20 msgr2=0x7f5cf80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5cf80a2f20 0x7f5cf80a53e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c199100 tx=0x7f5d08005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 -- 172.21.15.49:0/1196224847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c1075c0 msgr2=0x7f5d0c19f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c1075c0 0x7f5d0c19f1a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400eb10 tx=0x7f5d0400c2d0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 -- 172.21.15.49:0/1196224847 shutdown_connections 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f5cf80a2f20 0x7f5cf80a53e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d0c104c90 0x7f5d0c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d0c1075c0 0x7f5d0c19f1a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 --2- 172.21.15.49:0/1196224847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0c102360 0x7f5d0c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 -- 172.21.15.49:0/1196224847 >> 172.21.15.49:0/1196224847 conn(0x7f5d0c0fbd70 msgr2=0x7f5d0c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 -- 172.21.15.49:0/1196224847 shutdown_connections 2024-03-26T18:01:57.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.667+0000 7f5d13352640 1 -- 172.21.15.49:0/1196224847 wait complete. 2024-03-26T18:01:57.658 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:57 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1245648011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:57.658 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:57 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/824940072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:57.660 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.787+0000 7f6d04ba9640 1 -- 172.21.15.49:0/2211068422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d00102370 msgr2=0x7f6d00108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.787+0000 7f6d04ba9640 1 --2- 172.21.15.49:0/2211068422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d00102370 0x7f6d00108c00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009ca0 tx=0x7f6cf802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 -- 172.21.15.49:0/2211068422 shutdown_connections 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 --2- 172.21.15.49:0/2211068422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d001019b0 0x7f6d00101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 --2- 172.21.15.49:0/2211068422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d00102370 0x7f6d00108c00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 --2- 172.21.15.49:0/2211068422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d001007c0 0x7f6d00100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 -- 172.21.15.49:0/2211068422 >> 172.21.15.49:0/2211068422 conn(0x7f6d000fbd50 msgr2=0x7f6d000fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 -- 172.21.15.49:0/2211068422 shutdown_connections 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 -- 172.21.15.49:0/2211068422 wait complete. 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 Processor -- start 2024-03-26T18:01:57.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 -- start start 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d001007c0 0x7f6d0010b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.788+0000 7f6d04ba9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d001019b0 0x7f6d00109780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6d04ba9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d00102370 0x7f6d00109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6d04ba9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0010d560 con 0x7f6d001007c0 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6d04ba9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0010d3e0 con 0x7f6d00102370 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6d04ba9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0010d6e0 con 0x7f6d001019b0 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfe59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d001007c0 0x7f6d0010b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfe59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d001007c0 0x7f6d0010b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33538/0 (socket says 172.21.15.49:33538) 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfe59c640 1 -- 172.21.15.49:0/239093191 learned_addr learned my addr 172.21.15.49:0/239093191 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfdd9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d001019b0 0x7f6d00109780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfed9d640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d00102370 0x7f6d00109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfe59c640 1 -- 172.21.15.49:0/239093191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d001019b0 msgr2=0x7f6d00109780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d001019b0 0x7f6d00109780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfe59c640 1 -- 172.21.15.49:0/239093191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d00102370 msgr2=0x7f6d00109cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d00102370 0x7f6d00109cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfe59c640 1 -- 172.21.15.49:0/239093191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0010a550 con 0x7f6d001007c0 2024-03-26T18:01:57.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d001007c0 0x7f6d0010b140 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800e9c0 tx=0x7f6ce800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cf77fe640 1 -- 172.21.15.49:0/239093191 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800cde0 con 0x7f6d001007c0 2024-03-26T18:01:57.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cf77fe640 1 -- 172.21.15.49:0/239093191 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce8010910 con 0x7f6d001007c0 2024-03-26T18:01:57.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6d04ba9640 1 -- 172.21.15.49:0/239093191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d001a1cd0 con 0x7f6d001007c0 2024-03-26T18:01:57.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6cf77fe640 1 -- 172.21.15.49:0/239093191 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80189c0 con 0x7f6d001007c0 2024-03-26T18:01:57.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.789+0000 7f6d04ba9640 1 -- 172.21.15.49:0/239093191 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d001a21c0 con 0x7f6d001007c0 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.791+0000 7f6cf77fe640 1 -- 172.21.15.49:0/239093191 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f6ce8010ac0 con 0x7f6d001007c0 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6d07791740 1 Processor -- start 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6cf77fe640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6cec0a2fa0 0x7f6cec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6d07791740 1 -- start start 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6cf77fe640 1 -- 172.21.15.49:0/239093191 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f6ce80cfc70 con 0x7f6d001007c0 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6d07791740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df81e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6d07791740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df81f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6cfed9d640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6cec0a2fa0 0x7f6cec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6d07791740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df826060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6d07791740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5654df782270 con 0x5654df81f7d0 2024-03-26T18:01:57.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6d07791740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5654df72ac80 con 0x5654df81ee10 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.792+0000 7f6d07791740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5654df77e220 con 0x5654df81dc40 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df826060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfe59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df81e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df826060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33548/0 (socket says 172.21.15.49:33548) 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfed9d640 1 -- 172.21.15.49:0/2675637009 learned_addr learned my addr 172.21.15.49:0/2675637009 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfe59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df81e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:36910/0 (socket says 172.21.15.49:36910) 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/2675637009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df81f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfdd9b640 1 -- 172.21.15.49:0/2675637009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 msgr2=0x5654df81e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/2675637009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df81e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfdd9b640 1 -- 172.21.15.49:0/2675637009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 msgr2=0x5654df826060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/2675637009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df826060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfdd9b640 1 -- 172.21.15.49:0/2675637009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654df73cfb0 con 0x5654df81ee10 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/2675637009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df81e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfed9d640 1 --2- 172.21.15.49:0/2675637009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df826060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfed9d640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6cec0a2fa0 0x7f6cec0a5460 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009430 tx=0x7f6cf8009880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/2675637009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df81f290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009bd0 tx=0x7f6cf0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c985944b1b829de1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cf4ff9640 1 -- 172.21.15.49:0/2675637009 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0004230 con 0x5654df81ee10 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cf4ff9640 1 -- 172.21.15.49:0/2675637009 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf0009e40 con 0x5654df81ee10 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.793+0000 7f6cf4ff9640 1 -- 172.21.15.49:0/2675637009 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0004cc0 con 0x5654df81ee10 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 -- 172.21.15.49:0/2675637009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 msgr2=0x5654df81f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 --2- 172.21.15.49:0/2675637009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df81f290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009bd0 tx=0x7f6cf0011810 comp rx=0 tx=0).stop 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 -- 172.21.15.49:0/2675637009 shutdown_connections 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 --2- 172.21.15.49:0/2675637009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df826060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 --2- 172.21.15.49:0/2675637009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df81f290 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 --2- 172.21.15.49:0/2675637009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df81e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 -- 172.21.15.49:0/2675637009 >> 172.21.15.49:0/2675637009 conn(0x5654df819030 msgr2=0x5654df819430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 -- 172.21.15.49:0/2675637009 shutdown_connections 2024-03-26T18:01:57.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 -- 172.21.15.49:0/2675637009 wait complete. 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 -- start start 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.794+0000 7f6d07791740 1 Processor -- start 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6d07791740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df828120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6d07791740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df828660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6d07791740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df82f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6d07791740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5654df8267e0 con 0x5654df81f7d0 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6d07791740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5654df73dfc0 con 0x5654df81ee10 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6d07791740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5654df826960 con 0x5654df81dc40 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6d07791740 10 client.0 _ll_register_callbacks cb 0x5654df738770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df82f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df82f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33558/0 (socket says 172.21.15.49:33558) 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfed9d640 1 -- 172.21.15.49:0/168391837 learned_addr learned my addr 172.21.15.49:0/168391837 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df828660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df828120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfed9d640 1 -- 172.21.15.49:0/168391837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 msgr2=0x5654df828120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfed9d640 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df828120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfed9d640 1 -- 172.21.15.49:0/168391837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 msgr2=0x5654df828660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfed9d640 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df828660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfed9d640 1 -- 172.21.15.49:0/168391837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654df4962d0 con 0x5654df81f7d0 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6cfed9d640 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df82f6a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8005e30 tx=0x7f6cf802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6f4cdb0a81f271 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6ce5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/168391837 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8039040 con 0x5654df81f7d0 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/168391837 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f6cf8004440 con 0x5654df81f7d0 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.795+0000 7f6d07791740 1 -- 172.21.15.49:0/168391837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654df78f7b0 con 0x5654df81f7d0 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/168391837 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6cf805a050 con 0x5654df81f7d0 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6d07791740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:57.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6d07791740 1 -- 172.21.15.49:0/168391837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654df5606b0 con 0x5654df81f7d0 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/168391837 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5654df78f7b0 con 0x5654df81f7d0 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/168391837 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6cf803f040 con 0x5654df81f7d0 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6d07791740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6d07791740 10 client.15492 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6d07791740 20 client.15492 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6d07791740 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5654df834f30 0x5654df8363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6d07791740 4 client.15492 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6d07791740 1 -- 172.21.15.49:0/168391837 --> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654df789b00 con 0x5654df834f30 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.796+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5654df834f30 0x5654df8363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5654df834f30 0x5654df8363a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a77e73e01fffbe9 server_cookie=5e8fa252d596cc96 in_seq=0 out_seq=0 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6ce5ffb640 10 client.15492 ms_handle_connect on v2:172.21.15.115:6838/1435040687 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/168391837 <== mds.0 v2:172.21.15.115:6838/1435040687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5654df789b00 con 0x5654df834f30 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6ce5ffb640 10 client.15492 handle_command_reply: tid=0 2024-03-26T18:01:57.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6d07791740 1 client.15492 shutdown 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6d07791740 1 -- 172.21.15.49:0/168391837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 msgr2=0x5654df82f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6d07791740 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df82f6a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8005e30 tx=0x7f6cf802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6d07791740 1 -- 172.21.15.49:0/168391837 shutdown_connections 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6d07791740 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5654df834f30 0x5654df8363a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6d07791740 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df82f6a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6ce5ffb640 0 client.15492 ms_handle_reset on v2:172.21.15.115:6838/1435040687 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6d07791740 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df828660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6d07791740 1 --2- 172.21.15.49:0/168391837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df828120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.797+0000 7f6d07791740 1 -- 172.21.15.49:0/168391837 >> 172.21.15.49:0/168391837 conn(0x5654df819030 msgr2=0x5654df818230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.798+0000 7f6d07791740 1 -- 172.21.15.49:0/168391837 shutdown_connections 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.798+0000 7f6d07791740 1 -- 172.21.15.49:0/168391837 wait complete. 2024-03-26T18:01:57.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.798+0000 7f6d07791740 20 client.15492 trim_cache size 0 max 16384 2024-03-26T18:01:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.815+0000 7f6d07791740 1 Processor -- start 2024-03-26T18:01:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.815+0000 7f6d07791740 1 -- start start 2024-03-26T18:01:57.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6d07791740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df819410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6d07791740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df82f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6d07791740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df822920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6d07791740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5654df8346e0 con 0x5654df81f7d0 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6d07791740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5654df834db0 con 0x5654df81ee10 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6d07791740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5654df835480 con 0x5654df81dc40 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df822920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfdd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df82f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfdd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df82f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48844/0 (socket says 172.21.15.49:48844) 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfdd9b640 1 -- 172.21.15.49:0/3640726314 learned_addr learned my addr 172.21.15.49:0/3640726314 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df822920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33572/0 (socket says 172.21.15.49:33572) 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/3640726314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df819410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfdd9b640 1 -- 172.21.15.49:0/3640726314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 msgr2=0x5654df819410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/3640726314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df819410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfdd9b640 1 -- 172.21.15.49:0/3640726314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 msgr2=0x5654df822920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/3640726314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df822920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfdd9b640 1 -- 172.21.15.49:0/3640726314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654df81e890 con 0x5654df81ee10 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfed9d640 1 --2- 172.21.15.49:0/3640726314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df822920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/3640726314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df82f660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5654df819fa0 tx=0x7f6cf00096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93884575e0f1d8fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6ce7fff640 1 -- 172.21.15.49:0/3640726314 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf001e040 con 0x5654df81ee10 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6ce7fff640 1 -- 172.21.15.49:0/3640726314 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf0004510 con 0x5654df81ee10 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.816+0000 7f6ce7fff640 1 -- 172.21.15.49:0/3640726314 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf00246e0 con 0x5654df81ee10 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 -- 172.21.15.49:0/3640726314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 msgr2=0x5654df82f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 --2- 172.21.15.49:0/3640726314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df82f660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5654df819fa0 tx=0x7f6cf00096a0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 -- 172.21.15.49:0/3640726314 shutdown_connections 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 --2- 172.21.15.49:0/3640726314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81f7d0 0x5654df822920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 --2- 172.21.15.49:0/3640726314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81ee10 0x5654df82f660 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 --2- 172.21.15.49:0/3640726314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81dc40 0x5654df819410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 -- 172.21.15.49:0/3640726314 >> 172.21.15.49:0/3640726314 conn(0x5654df819030 msgr2=0x5654df813e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 -- 172.21.15.49:0/3640726314 shutdown_connections 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 -- 172.21.15.49:0/3640726314 wait complete. 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 -- start start 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 Processor -- start 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81dc40 0x5654df82af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81ee10 0x5654df82b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81f7d0 0x5654df82e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.817+0000 7f6d07791740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5654df836220 con 0x5654df81ee10 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6d07791740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x5654df835b50 con 0x5654df81dc40 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6d07791740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5654df8368f0 con 0x5654df81f7d0 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6d07791740 10 client.0 _ll_register_callbacks cb 0x7f6cf0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81dc40 0x5654df82af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81ee10 0x5654df82b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81dc40 0x5654df82af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:48856/0 (socket says 172.21.15.49:48856) 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfed9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81f7d0 0x5654df82e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfe59c640 1 -- 172.21.15.49:0/2060068077 learned_addr learned my addr 172.21.15.49:0/2060068077 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81ee10 0x5654df82b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33582/0 (socket says 172.21.15.49:33582) 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfe59c640 1 -- 172.21.15.49:0/2060068077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81f7d0 msgr2=0x5654df82e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81f7d0 0x5654df82e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfe59c640 1 -- 172.21.15.49:0/2060068077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81ee10 msgr2=0x5654df82b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81ee10 0x5654df82b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfe59c640 1 -- 172.21.15.49:0/2060068077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654df82ea50 con 0x5654df81dc40 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81ee10 0x5654df82b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6cfe59c640 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81dc40 0x5654df82af00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ce809bf50 tx=0x7f6ce80bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df114ae8545a64e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6ce5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/2060068077 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80bc0b0 con 0x5654df81dc40 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6d07791740 1 -- 172.21.15.49:0/2060068077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf0004510 con 0x5654df81dc40 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/2060068077 <== mon.2 v2:172.21.15.115:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f6ce8098630 con 0x5654df81dc40 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/2060068077 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ce8018b80 con 0x5654df81dc40 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6d07791740 10 client.25394 fetch_fsmap learned FSMap version 28 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.818+0000 7f6d07791740 1 -- 172.21.15.49:0/2060068077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654df89aed0 con 0x5654df81dc40 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/2060068077 <== mon.2 v2:172.21.15.115:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf0004510 con 0x5654df81dc40 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/2060068077 <== mon.2 v2:172.21.15.115:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6ce80b6070 con 0x5654df81dc40 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6d07791740 10 client.25394 fetch_fsmap finished waiting for FSMap version 28 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6d07791740 10 client.25394 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6d07791740 20 client.25394 populate_metadata read hostname 'smithi049' 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6d07791740 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5654df89fb10 0x5654df89e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6d07791740 4 client.25394 mds_command: new command op to 24533 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6d07791740 1 -- 172.21.15.49:0/2060068077 --> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] -- command(tid 0: {"prefix": "status"}) -- 0x5654df707a00 con 0x5654df89fb10 2024-03-26T18:01:57.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5654df89fb10 0x5654df89e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6cfdd9b640 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5654df89fb10 0x5654df89e820 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467d400977faae26 server_cookie=9b0820d3f92863d9 in_seq=0 out_seq=0 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6ce5ffb640 10 client.25394 ms_handle_connect on v2:172.21.15.115:6838/1435040687 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6ce5ffb640 1 -- 172.21.15.49:0/2060068077 <== mds.0 v2:172.21.15.115:6838/1435040687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5654df707a00 con 0x5654df89fb10 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.819+0000 7f6ce5ffb640 10 client.25394 handle_command_reply: tid=0 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 client.25394 shutdown 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 -- 172.21.15.49:0/2060068077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81dc40 msgr2=0x5654df82af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81dc40 0x5654df82af00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ce809bf50 tx=0x7f6ce80bc040 comp rx=0 tx=0).stop 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 -- 172.21.15.49:0/2060068077 shutdown_connections 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.115:6838/1435040687,v1:172.21.15.115:6839/1435040687] conn(0x5654df89fb10 0x5654df89e820 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654df81f7d0 0x5654df82e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6ce5ffb640 0 client.25394 ms_handle_reset on v2:172.21.15.115:6838/1435040687 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5654df81ee10 0x5654df82b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 --2- 172.21.15.49:0/2060068077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5654df81dc40 0x5654df82af00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 -- 172.21.15.49:0/2060068077 >> 172.21.15.49:0/2060068077 conn(0x5654df819030 msgr2=0x5654df825b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 -- 172.21.15.49:0/2060068077 shutdown_connections 2024-03-26T18:01:57.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 1 -- 172.21.15.49:0/2060068077 wait complete. 2024-03-26T18:01:57.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d07791740 20 client.25394 trim_cache size 0 max 16384 2024-03-26T18:01:57.809 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:01:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "5a3632a4-eb99-11ee-95d0-87774f69a715", 2024-03-26T18:01:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-03-26T18:01:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24533, 2024-03-26T18:01:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-03-26T18:01:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-03-26T18:01:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-03-26T18:01:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 130.39268669200001, 2024-03-26T18:01:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2024-03-26T18:01:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 94, 2024-03-26T18:01:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T18:01:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 134.51972120900001 2024-03-26T18:01:57.810 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:01:57.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 -- 172.21.15.49:0/239093191 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6cec0a2fa0 msgr2=0x7f6cec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6cec0a2fa0 0x7f6cec0a5460 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009430 tx=0x7f6cf8009880 comp rx=0 tx=0).stop 2024-03-26T18:01:57.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 -- 172.21.15.49:0/239093191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d001007c0 msgr2=0x7f6d0010b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d001007c0 0x7f6d0010b140 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800e9c0 tx=0x7f6ce800ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:57.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 -- 172.21.15.49:0/239093191 shutdown_connections 2024-03-26T18:01:57.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6cec0a2fa0 0x7f6cec0a5460 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6d001019b0 0x7f6d00109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d00102370 0x7f6d00109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 --2- 172.21.15.49:0/239093191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d001007c0 0x7f6d0010b140 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 -- 172.21.15.49:0/239093191 >> 172.21.15.49:0/239093191 conn(0x7f6d000fbd50 msgr2=0x7f6d00104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 -- 172.21.15.49:0/239093191 shutdown_connections 2024-03-26T18:01:57.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.820+0000 7f6d04ba9640 1 -- 172.21.15.49:0/239093191 wait complete. 2024-03-26T18:01:57.813 INFO:teuthology.run_tasks:Running task sequential... 2024-03-26T18:01:57.823 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-26T18:01:57.823 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-26T18:01:57.824 INFO:teuthology.task.print:Enabling mgr modules 2024-03-26T18:01:57.824 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-03-26T18:01:57.824 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T18:01:57.825 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T18:01:57.825 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-26T18:01:57.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:57 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1245648011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:57.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:57 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/824940072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:57.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.950+0000 7f6db4b62640 1 -- 172.21.15.49:0/2590381088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6db0100830 msgr2=0x7f6db0100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.950+0000 7f6db4b62640 1 --2- 172.21.15.49:0/2590381088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6db0100830 0x7f6db0100c50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009860 tx=0x7f6da002bb70 comp rx=0 tx=0).stop 2024-03-26T18:01:57.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.950+0000 7f6db4b62640 1 -- 172.21.15.49:0/2590381088 shutdown_connections 2024-03-26T18:01:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.950+0000 7f6db4b62640 1 --2- 172.21.15.49:0/2590381088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6db01023e0 0x7f6db0108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.950+0000 7f6db4b62640 1 --2- 172.21.15.49:0/2590381088 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6db0101a20 0x7f6db0101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.950+0000 7f6db4b62640 1 --2- 172.21.15.49:0/2590381088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6db0100830 0x7f6db0100c50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.950+0000 7f6db4b62640 1 -- 172.21.15.49:0/2590381088 >> 172.21.15.49:0/2590381088 conn(0x7f6db00fbda0 msgr2=0x7f6db00fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.950+0000 7f6db4b62640 1 -- 172.21.15.49:0/2590381088 shutdown_connections 2024-03-26T18:01:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.950+0000 7f6db4b62640 1 -- 172.21.15.49:0/2590381088 wait complete. 2024-03-26T18:01:57.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6db4b62640 1 Processor -- start 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6db4b62640 1 -- start start 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6db4b62640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6db0100830 0x7f6db0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6db4b62640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6db0101a20 0x7f6db01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6db4b62640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6db01023e0 0x7f6db019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6db4b62640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6db010d5d0 con 0x7f6db01023e0 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6db4b62640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6db010d450 con 0x7f6db0101a20 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6db4b62640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6db010d750 con 0x7f6db0100830 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6daed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6db01023e0 0x7f6db019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6daed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6db01023e0 0x7f6db019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:33608/0 (socket says 172.21.15.49:33608) 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6db0101a20 0x7f6db01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6daed9d640 1 -- 172.21.15.49:0/680587961 learned_addr learned my addr 172.21.15.49:0/680587961 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6dae59c640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6db0100830 0x7f6db0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6daed9d640 1 -- 172.21.15.49:0/680587961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6db0100830 msgr2=0x7f6db0197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.951+0000 7f6daed9d640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6db0100830 0x7f6db0197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.952+0000 7f6daed9d640 1 -- 172.21.15.49:0/680587961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6db0101a20 msgr2=0x7f6db01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.952+0000 7f6daed9d640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6db0101a20 0x7f6db01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.952+0000 7f6daed9d640 1 -- 172.21.15.49:0/680587961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db019f7a0 con 0x7f6db01023e0 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.952+0000 7f6da7fff640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6db0101a20 0x7f6db01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.952+0000 7f6daed9d640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6db01023e0 0x7f6db019f130 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6da800eb10 tx=0x7f6da800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.952+0000 7f6da77fe640 1 -- 172.21.15.49:0/680587961 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da800ce20 con 0x7f6db01023e0 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.952+0000 7f6db4b62640 1 -- 172.21.15.49:0/680587961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db019fa90 con 0x7f6db01023e0 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.952+0000 7f6da77fe640 1 -- 172.21.15.49:0/680587961 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da8005020 con 0x7f6db01023e0 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.952+0000 7f6da77fe640 1 -- 172.21.15.49:0/680587961 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8010690 con 0x7f6db01023e0 2024-03-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.952+0000 7f6db4b62640 1 -- 172.21.15.49:0/680587961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db019ffe0 con 0x7f6db01023e0 2024-03-26T18:01:57.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.953+0000 7f6db4b62640 1 -- 172.21.15.49:0/680587961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db0107b50 con 0x7f6db01023e0 2024-03-26T18:01:57.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.954+0000 7f6da77fe640 1 -- 172.21.15.49:0/680587961 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) ==== 137854+0+0 (secure 0 0 0) 0x7f6da8005540 con 0x7f6db01023e0 2024-03-26T18:01:57.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.955+0000 7f6da77fe640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6d9c0a2fc0 0x7f6d9c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:57.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.955+0000 7f6dae59c640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6d9c0a2fc0 0x7f6d9c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:57.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.955+0000 7f6da77fe640 1 -- 172.21.15.49:0/680587961 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f6da80cfee0 con 0x7f6db01023e0 2024-03-26T18:01:57.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.955+0000 7f6dae59c640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6d9c0a2fc0 0x7f6d9c0a5480 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009830 tx=0x7f6da0034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:57.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:57.958+0000 7f6da77fe640 1 -- 172.21.15.49:0/680587961 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6da8053740 con 0x7f6db01023e0 2024-03-26T18:01:57.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:57 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1245648011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:57.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:57 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/824940072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:01:58.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.192+0000 7f6db4b62640 1 -- 172.21.15.49:0/680587961 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f6db002d240 con 0x7f6db01023e0 2024-03-26T18:01:58.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.556+0000 7f6da77fe640 1 -- 172.21.15.49:0/680587961 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f6da809c990 con 0x7f6db01023e0 2024-03-26T18:01:58.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 -- 172.21.15.49:0/680587961 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6d9c0a2fc0 msgr2=0x7f6d9c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:58.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6d9c0a2fc0 0x7f6d9c0a5480 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009830 tx=0x7f6da0034ad0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 -- 172.21.15.49:0/680587961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6db01023e0 msgr2=0x7f6db019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:58.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6db01023e0 0x7f6db019f130 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6da800eb10 tx=0x7f6da800ef10 comp rx=0 tx=0).stop 2024-03-26T18:01:58.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 -- 172.21.15.49:0/680587961 shutdown_connections 2024-03-26T18:01:58.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6d9c0a2fc0 0x7f6d9c0a5480 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6db01023e0 0x7f6db019f130 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6db0101a20 0x7f6db01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 --2- 172.21.15.49:0/680587961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6db0100830 0x7f6db0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 -- 172.21.15.49:0/680587961 >> 172.21.15.49:0/680587961 conn(0x7f6db00fbda0 msgr2=0x7f6db0104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:58.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.559+0000 7f6db4b62640 1 -- 172.21.15.49:0/680587961 shutdown_connections 2024-03-26T18:01:58.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.560+0000 7f6db4b62640 1 -- 172.21.15.49:0/680587961 wait complete. 2024-03-26T18:01:58.560 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.694+0000 7f08886dc640 1 -- 172.21.15.49:0/630499046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08801014b0 msgr2=0x7f0880101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.694+0000 7f08886dc640 1 --2- 172.21.15.49:0/630499046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08801014b0 0x7f0880101930 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0878009ca0 tx=0x7f087802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.694+0000 7f08886dc640 1 -- 172.21.15.49:0/630499046 shutdown_connections 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.694+0000 7f08886dc640 1 --2- 172.21.15.49:0/630499046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08801014b0 0x7f0880101930 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.694+0000 7f08886dc640 1 --2- 172.21.15.49:0/630499046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0880100b10 0x7f0880100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.694+0000 7f08886dc640 1 --2- 172.21.15.49:0/630499046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08801001b0 0x7f08801005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.694+0000 7f08886dc640 1 -- 172.21.15.49:0/630499046 >> 172.21.15.49:0/630499046 conn(0x7f08800fbd50 msgr2=0x7f08800fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f08886dc640 1 -- 172.21.15.49:0/630499046 shutdown_connections 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f08886dc640 1 -- 172.21.15.49:0/630499046 wait complete. 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f08886dc640 1 Processor -- start 2024-03-26T18:01:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f08886dc640 1 -- start start 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f08886dc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08801001b0 0x7f0880199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f08886dc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0880100b10 0x7f088019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f08886dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08801014b0 0x7f08801a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f08886dc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f088010f730 con 0x7f08801001b0 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f08886dc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f088010f5b0 con 0x7f08801014b0 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f08886dc640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f088010f8b0 con 0x7f0880100b10 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f0886478640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08801001b0 0x7f0880199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f0886478640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08801001b0 0x7f0880199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52882/0 (socket says 172.21.15.49:52882) 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.695+0000 7f0886478640 1 -- 172.21.15.49:0/516336361 learned_addr learned my addr 172.21.15.49:0/516336361 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f0885c77640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0880100b10 0x7f088019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f0886c79640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08801014b0 0x7f08801a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f0886478640 1 -- 172.21.15.49:0/516336361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0880100b10 msgr2=0x7f088019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f0886478640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0880100b10 0x7f088019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f0886478640 1 -- 172.21.15.49:0/516336361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08801014b0 msgr2=0x7f08801a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:58.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f0886478640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08801014b0 0x7f08801a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f0886478640 1 -- 172.21.15.49:0/516336361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08801a1920 con 0x7f08801001b0 2024-03-26T18:01:58.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f0886478640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08801001b0 0x7f0880199d10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f087c00d950 tx=0x7f087c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:58.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f08777fe640 1 -- 172.21.15.49:0/516336361 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c017030 con 0x7f08801001b0 2024-03-26T18:01:58.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f08777fe640 1 -- 172.21.15.49:0/516336361 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f087c004020 con 0x7f08801001b0 2024-03-26T18:01:58.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f08886dc640 1 -- 172.21.15.49:0/516336361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08801a1c10 con 0x7f08801001b0 2024-03-26T18:01:58.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f08777fe640 1 -- 172.21.15.49:0/516336361 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c0047a0 con 0x7f08801001b0 2024-03-26T18:01:58.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.696+0000 7f08886dc640 1 -- 172.21.15.49:0/516336361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08801a21b0 con 0x7f08801001b0 2024-03-26T18:01:58.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.697+0000 7f08886dc640 1 -- 172.21.15.49:0/516336361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088010af00 con 0x7f08801001b0 2024-03-26T18:01:58.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.698+0000 7f08777fe640 1 -- 172.21.15.49:0/516336361 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) ==== 137871+0+0 (secure 0 0 0) 0x7f087c0041d0 con 0x7f08801001b0 2024-03-26T18:01:58.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.699+0000 7f08777fe640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f086c0a3090 0x7f086c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:58.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.699+0000 7f0885c77640 1 -- 172.21.15.49:0/516336361 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f086c0a3090 msgr2=0x7f086c0a5550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/3127469495 2024-03-26T18:01:58.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.699+0000 7f08777fe640 1 -- 172.21.15.49:0/516336361 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f087c0cf190 con 0x7f08801001b0 2024-03-26T18:01:58.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.699+0000 7f0885c77640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f086c0a3090 0x7f086c0a5550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:01:58.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.704+0000 7f08777fe640 1 -- 172.21.15.49:0/516336361 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f087c09bc40 con 0x7f08801001b0 2024-03-26T18:01:58.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:58 smithi171 ceph-mon[36033]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-26T18:01:58.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:58 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/680587961' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-26T18:01:58.856 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:01:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: ignoring --setuser ceph since I am not root 2024-03-26T18:01:58.856 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:01:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: ignoring --setgroup ceph since I am not root 2024-03-26T18:01:58.856 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:01:58 smithi171 ceph-mgr[37825]: -- 172.21.15.171:0/923403454 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e45ff05a0 con 0x555e45fc5000 2024-03-26T18:01:58.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.889+0000 7f08886dc640 1 -- 172.21.15.49:0/516336361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f088002d240 con 0x7f08801001b0 2024-03-26T18:01:58.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.892+0000 7f08777fe640 1 -- 172.21.15.49:0/516336361 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) ==== 138+0+0 (secure 0 0 0) 0x7f087c09b540 con 0x7f08801001b0 2024-03-26T18:01:58.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 -- 172.21.15.49:0/516336361 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f086c0a3090 msgr2=0x7f086c0a5550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f086c0a3090 0x7f086c0a5550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 -- 172.21.15.49:0/516336361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08801001b0 msgr2=0x7f0880199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08801001b0 0x7f0880199d10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f087c00d950 tx=0x7f087c00de10 comp rx=0 tx=0).stop 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 -- 172.21.15.49:0/516336361 shutdown_connections 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f086c0a3090 0x7f086c0a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0880100b10 0x7f088019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08801014b0 0x7f08801a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 --2- 172.21.15.49:0/516336361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08801001b0 0x7f0880199d10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 -- 172.21.15.49:0/516336361 >> 172.21.15.49:0/516336361 conn(0x7f08800fbd50 msgr2=0x7f08800fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.894+0000 7f08886dc640 1 -- 172.21.15.49:0/516336361 shutdown_connections 2024-03-26T18:01:58.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:58.895+0000 7f08886dc640 1 -- 172.21.15.49:0/516336361 wait complete. 2024-03-26T18:01:58.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:58 smithi049 ceph-mon[32331]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-26T18:01:58.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:58 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/680587961' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-26T18:01:58.887 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:01:58 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: ignoring --setuser ceph since I am not root 2024-03-26T18:01:58.887 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:01:58 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: ignoring --setgroup ceph since I am not root 2024-03-26T18:01:58.888 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-26T18:01:58.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:58 smithi115 ceph-mon[36174]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-26T18:01:58.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:58 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/680587961' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-26T18:01:58.987 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:01:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: ignoring --setuser ceph since I am not root 2024-03-26T18:01:58.987 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:01:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: ignoring --setgroup ceph since I am not root 2024-03-26T18:01:58.987 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:01:58 smithi115 ceph-mgr[37560]: -- 172.21.15.115:0/1481665825 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be86f925a0 con 0x55be86f67000 2024-03-26T18:01:58.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.016+0000 7f6c916c6640 1 -- 172.21.15.49:0/98006036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c1007e0 msgr2=0x7f6c8c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:58.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.016+0000 7f6c916c6640 1 --2- 172.21.15.49:0/98006036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c1007e0 0x7f6c8c100c00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009900 tx=0x7f6c8002bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:58.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 -- 172.21.15.49:0/98006036 shutdown_connections 2024-03-26T18:01:58.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 --2- 172.21.15.49:0/98006036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c8c102390 0x7f6c8c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 --2- 172.21.15.49:0/98006036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c1019d0 0x7f6c8c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 --2- 172.21.15.49:0/98006036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c1007e0 0x7f6c8c100c00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:58.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 -- 172.21.15.49:0/98006036 >> 172.21.15.49:0/98006036 conn(0x7f6c8c0fbd50 msgr2=0x7f6c8c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:59.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 -- 172.21.15.49:0/98006036 shutdown_connections 2024-03-26T18:01:59.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 -- 172.21.15.49:0/98006036 wait complete. 2024-03-26T18:01:59.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 Processor -- start 2024-03-26T18:01:59.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 -- start start 2024-03-26T18:01:59.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c1007e0 0x7f6c8c104840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:59.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.017+0000 7f6c916c6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c1019d0 0x7f6c8c102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:59.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c916c6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c8c102390 0x7f6c8c1033c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:59.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c916c6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10c460 con 0x7f6c8c1019d0 2024-03-26T18:01:59.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c916c6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10c2e0 con 0x7f6c8c1007e0 2024-03-26T18:01:59.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c916c6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10c5e0 con 0x7f6c8c102390 2024-03-26T18:01:59.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c1019d0 0x7f6c8c102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:59.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c1007e0 0x7f6c8c104840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:59.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c8c102390 0x7f6c8c1033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c1019d0 0x7f6c8c102e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52902/0 (socket says 172.21.15.49:52902) 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c1007e0 0x7f6c8c104840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34784/0 (socket says 172.21.15.49:34784) 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8a7fc640 1 -- 172.21.15.49:0/4026974422 learned_addr learned my addr 172.21.15.49:0/4026974422 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8b7fe640 1 -- 172.21.15.49:0/4026974422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c1007e0 msgr2=0x7f6c8c104840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8b7fe640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c1007e0 0x7f6c8c104840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8b7fe640 1 -- 172.21.15.49:0/4026974422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c1019d0 msgr2=0x7f6c8c102e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8b7fe640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c1019d0 0x7f6c8c102e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8b7fe640 1 -- 172.21.15.49:0/4026974422 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c103c50 con 0x7f6c8c102390 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8a7fc640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c1019d0 0x7f6c8c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c8b7fe640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c8c102390 0x7f6c8c1033c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009fd0 tx=0x7f6c8400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.018+0000 7f6c73fff640 1 -- 172.21.15.49:0/4026974422 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400ce20 con 0x7f6c8c102390 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.019+0000 7f6c916c6640 1 -- 172.21.15.49:0/4026974422 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c1a1cc0 con 0x7f6c8c102390 2024-03-26T18:01:59.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.019+0000 7f6c73fff640 1 -- 172.21.15.49:0/4026974422 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c84005020 con 0x7f6c8c102390 2024-03-26T18:01:59.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.019+0000 7f6c73fff640 1 -- 172.21.15.49:0/4026974422 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84010720 con 0x7f6c8c102390 2024-03-26T18:01:59.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.019+0000 7f6c916c6640 1 -- 172.21.15.49:0/4026974422 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8c1a21b0 con 0x7f6c8c102390 2024-03-26T18:01:59.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.020+0000 7f6c916c6640 1 -- 172.21.15.49:0/4026974422 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c8c1a24a0 con 0x7f6c8c102390 2024-03-26T18:01:59.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.021+0000 7f6c73fff640 1 -- 172.21.15.49:0/4026974422 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 19) ==== 137871+0+0 (secure 0 0 0) 0x7f6c84005540 con 0x7f6c8c102390 2024-03-26T18:01:59.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.021+0000 7f6c73fff640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6c740a3010 0x7f6c740a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:59.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.021+0000 7f6c8affd640 1 -- 172.21.15.49:0/4026974422 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6c740a3010 msgr2=0x7f6c740a54d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/3127469495 2024-03-26T18:01:59.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.021+0000 7f6c73fff640 1 -- 172.21.15.49:0/4026974422 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f6c840cfe30 con 0x7f6c8c102390 2024-03-26T18:01:59.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.021+0000 7f6c8affd640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6c740a3010 0x7f6c740a54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:01:59.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.025+0000 7f6c73fff640 1 -- 172.21.15.49:0/4026974422 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6c8409c8e0 con 0x7f6c8c102390 2024-03-26T18:01:59.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.210+0000 7f6c916c6640 1 -- 172.21.15.49:0/4026974422 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f6c8c1a2670 con 0x7f6c8c102390 2024-03-26T18:01:59.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.219+0000 7f6c73fff640 1 -- 172.21.15.49:0/4026974422 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7f6c8409c1e0 con 0x7f6c8c102390 2024-03-26T18:01:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 -- 172.21.15.49:0/4026974422 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6c740a3010 msgr2=0x7f6c740a54d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:01:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6c740a3010 0x7f6c740a54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 -- 172.21.15.49:0/4026974422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c8c102390 msgr2=0x7f6c8c1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c8c102390 0x7f6c8c1033c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009fd0 tx=0x7f6c8400ef10 comp rx=0 tx=0).stop 2024-03-26T18:01:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 -- 172.21.15.49:0/4026974422 shutdown_connections 2024-03-26T18:01:59.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f6c740a3010 0x7f6c740a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c8c102390 0x7f6c8c1033c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c8c1019d0 0x7f6c8c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 --2- 172.21.15.49:0/4026974422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c8c1007e0 0x7f6c8c104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 -- 172.21.15.49:0/4026974422 >> 172.21.15.49:0/4026974422 conn(0x7f6c8c0fbd50 msgr2=0x7f6c8c068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 -- 172.21.15.49:0/4026974422 shutdown_connections 2024-03-26T18:01:59.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:01:59.221+0000 7f6c916c6640 1 -- 172.21.15.49:0/4026974422 wait complete. 2024-03-26T18:01:59.214 INFO:teuthology.run_tasks:Running task kclient... 2024-03-26T18:01:59.227 INFO:tasks.kclient:Mounting kernel clients... 2024-03-26T18:01:59.228 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-26T18:01:59.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:01:59.228 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-03-26T18:01:59.243 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-26T18:01:59.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:01:59.244 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T18:01:59.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:01:59.316 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-03-26T18:01:59.405 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:01:59 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:01:59.164+0000 7f353b818200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T18:01:59.405 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:01:59 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:01:59.265+0000 7f353b818200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T18:01:59.406 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-03-26T18:01:59.407 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-26T18:01:59.407 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-03-26T18:01:59.407 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-03-26T18:01:59.407 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-26T18:01:59.553 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:01:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:01:59.257+0000 7fe33c897200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T18:01:59.553 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:01:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:01:59.398+0000 7fe33c897200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T18:01:59.555 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:01:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:01:59.194+0000 7f6c9342c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T18:01:59.555 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:01:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:01:59.291+0000 7f6c9342c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T18:01:59.674 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:01:59.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:59 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/680587961' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-26T18:01:59.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:59 smithi049 ceph-mon[32331]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-26T18:01:59.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:59 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/516336361' entity='client.admin' 2024-03-26T18:01:59.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:01:59 smithi049 ceph-mon[32331]: from='client.? ' entity='client.admin' 2024-03-26T18:01:59.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:59 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/680587961' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-26T18:01:59.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:59 smithi171 ceph-mon[36033]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-26T18:01:59.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:59 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/516336361' entity='client.admin' 2024-03-26T18:01:59.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:01:59 smithi171 ceph-mon[36033]: from='client.? ' entity='client.admin' 2024-03-26T18:01:59.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:59 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/680587961' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-26T18:01:59.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:59 smithi115 ceph-mon[36174]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-26T18:01:59.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:59 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/516336361' entity='client.admin' 2024-03-26T18:01:59.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:01:59 smithi115 ceph-mon[36174]: from='client.? ' entity='client.admin' 2024-03-26T18:02:00.052 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:01:59 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:01:59.984+0000 7f353b818200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T18:02:00.316 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:00.084+0000 7f353b818200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T18:02:00.317 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:00.234+0000 7f353b818200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T18:02:00.317 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:00.334+0000 7f353b818200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T18:02:00.336 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:01:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:01:59.953+0000 7f6c9342c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T18:02:00.336 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:00.048+0000 7f6c9342c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T18:02:00.336 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:00.199+0000 7f6c9342c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T18:02:00.336 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:00.354+0000 7f6c9342c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T18:02:00.403 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:00.103+0000 7fe33c897200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T18:02:00.403 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:00.237+0000 7fe33c897200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T18:02:00.403 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:00.422+0000 7fe33c897200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T18:02:00.605 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:00.472+0000 7f6c9342c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T18:02:00.685 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:00.421+0000 7f353b818200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T18:02:00.736 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:00.519+0000 7fe33c897200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T18:02:00.736 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:00.614+0000 7fe33c897200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T18:02:00.993 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:00.703+0000 7f353b818200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T18:02:00.993 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:00.817+0000 7f353b818200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T18:02:00.993 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:00.908+0000 7f353b818200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T18:02:01.105 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:00.811+0000 7f6c9342c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T18:02:01.105 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:00.946+0000 7f6c9342c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T18:02:01.105 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:01.046+0000 7f6c9342c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T18:02:01.151 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:00.919+0000 7fe33c897200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T18:02:01.151 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:01.050+0000 7fe33c897200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T18:02:01.251 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:01 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:01.010+0000 7f353b818200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T18:02:01.251 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:01 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:01.196+0000 7f353b818200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T18:02:01.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.418+0000 7f562ba43700 1 -- 172.21.15.49:0/3152702030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562406ab50 msgr2=0x7f562406af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:01.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.418+0000 7f562ba43700 1 --2- 172.21.15.49:0/3152702030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562406ab50 0x7f562406af30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f56200099c0 tx=0x7f5620009cd0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.418+0000 7f562ba43700 1 -- 172.21.15.49:0/3152702030 shutdown_connections 2024-03-26T18:02:01.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.418+0000 7f562ba43700 1 --2- 172.21.15.49:0/3152702030 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5624107730 0x7f562410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.418+0000 7f562ba43700 1 --2- 172.21.15.49:0/3152702030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562406ab50 0x7f562406af30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.418+0000 7f562ba43700 1 --2- 172.21.15.49:0/3152702030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f562406b500 0x7f5624107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.418+0000 7f562ba43700 1 -- 172.21.15.49:0/3152702030 >> 172.21.15.49:0/3152702030 conn(0x7f5624077300 msgr2=0x7f5624077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:01.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.418+0000 7f562ba43700 1 -- 172.21.15.49:0/3152702030 shutdown_connections 2024-03-26T18:02:01.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.418+0000 7f562ba43700 1 -- 172.21.15.49:0/3152702030 wait complete. 2024-03-26T18:02:01.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f562ba43700 1 Processor -- start 2024-03-26T18:02:01.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f562ba43700 1 -- start start 2024-03-26T18:02:01.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f562ba43700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f562406ab50 0x7f562419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:01.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f562ba43700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562406b500 0x7f562419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:01.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f562ba43700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5624107730 0x7f5624198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:01.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f562ba43700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56241126c0 con 0x7f562406b500 2024-03-26T18:02:01.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f562ba43700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112540 con 0x7f562406ab50 2024-03-26T18:02:01.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f562ba43700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112840 con 0x7f5624107730 2024-03-26T18:02:01.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f5628fde700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562406b500 0x7f562419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:01.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f56297df700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f562406ab50 0x7f562419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:01.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f5628fde700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562406b500 0x7f562419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52912/0 (socket says 172.21.15.49:52912) 2024-03-26T18:02:01.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f5629fe0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5624107730 0x7f5624198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:01.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.419+0000 7f5628fde700 1 -- 172.21.15.49:0/3975908722 learned_addr learned my addr 172.21.15.49:0/3975908722 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:02:01.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f5629fe0700 1 -- 172.21.15.49:0/3975908722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f562406ab50 msgr2=0x7f562419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:01.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f5629fe0700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f562406ab50 0x7f562419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f5629fe0700 1 -- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562406b500 msgr2=0x7f562419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:01.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f5629fe0700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562406b500 0x7f562419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f5629fe0700 1 -- 172.21.15.49:0/3975908722 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5624199160 con 0x7f5624107730 2024-03-26T18:02:01.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f5628fde700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562406b500 0x7f562419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:01.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f5629fe0700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5624107730 0x7f5624198a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f561c00efc0 tx=0x7f561c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:01.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f561a7fc700 1 -- 172.21.15.49:0/3975908722 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0092e0 con 0x7f5624107730 2024-03-26T18:02:01.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f561a7fc700 1 -- 172.21.15.49:0/3975908722 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561c004bb0 con 0x7f5624107730 2024-03-26T18:02:01.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f561a7fc700 1 -- 172.21.15.49:0/3975908722 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c004280 con 0x7f5624107730 2024-03-26T18:02:01.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f562ba43700 1 -- 172.21.15.49:0/3975908722 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56241993f0 con 0x7f5624107730 2024-03-26T18:02:01.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.420+0000 7f562ba43700 1 -- 172.21.15.49:0/3975908722 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56241a5710 con 0x7f5624107730 2024-03-26T18:02:01.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.421+0000 7f562ba43700 1 -- 172.21.15.49:0/3975908722 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562410a900 con 0x7f5624107730 2024-03-26T18:02:01.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.422+0000 7f561a7fc700 1 -- 172.21.15.49:0/3975908722 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 19) ==== 137871+0+0 (secure 0 0 0) 0x7f561c01b030 con 0x7f5624107730 2024-03-26T18:02:01.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.423+0000 7f561a7fc700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56100a3080 0x7f56100a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:01.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.423+0000 7f56297df700 1 -- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56100a3080 msgr2=0x7f56100a5540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/3127469495 2024-03-26T18:02:01.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.423+0000 7f561a7fc700 1 -- 172.21.15.49:0/3975908722 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f561c0cf690 con 0x7f5624107730 2024-03-26T18:02:01.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.423+0000 7f56297df700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56100a3080 0x7f56100a5540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:02:01.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.426+0000 7f561a7fc700 1 -- 172.21.15.49:0/3975908722 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f561c022020 con 0x7f5624107730 2024-03-26T18:02:01.479 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:01.170+0000 7fe33c897200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T18:02:01.479 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:01.291+0000 7fe33c897200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T18:02:01.479 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:01.498+0000 7fe33c897200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T18:02:01.605 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:01.171+0000 7f6c9342c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T18:02:01.605 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:01.369+0000 7f6c9342c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T18:02:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.624+0000 7f56297df700 1 -- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56100a3080 msgr2=0x7f56100a5540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/3127469495 2024-03-26T18:02:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.624+0000 7f56297df700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56100a3080 0x7f56100a5540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T18:02:01.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.726+0000 7f562ba43700 1 -- 172.21.15.49:0/3975908722 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5624199f30 con 0x7f5624107730 2024-03-26T18:02:01.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.726+0000 7f561a7fc700 1 -- 172.21.15.49:0/3975908722 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f561c010040 con 0x7f5624107730 2024-03-26T18:02:01.709 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-26T18:02:01.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 -- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56100a3080 msgr2=0x7f56100a5540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:02:01.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56100a3080 0x7f56100a5540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 -- 172.21.15.49:0/3975908722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5624107730 msgr2=0x7f5624198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:01.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5624107730 0x7f5624198a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f561c00efc0 tx=0x7f561c00c3a0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 -- 172.21.15.49:0/3975908722 shutdown_connections 2024-03-26T18:02:01.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f56100a3080 0x7f56100a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5624107730 0x7f5624198a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f562406ab50 0x7f562419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 --2- 172.21.15.49:0/3975908722 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f562406b500 0x7f562419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:01.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 -- 172.21.15.49:0/3975908722 >> 172.21.15.49:0/3975908722 conn(0x7f5624077300 msgr2=0x7f56240ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:01.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 -- 172.21.15.49:0/3975908722 shutdown_connections 2024-03-26T18:02:01.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:01.728+0000 7f562ba43700 1 -- 172.21.15.49:0/3975908722 wait complete. 2024-03-26T18:02:02.001 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:01 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3975908722' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T18:02:02.105 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:01 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3975908722' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T18:02:02.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:01 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3975908722' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T18:02:02.354 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:02 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:02.234+0000 7f353b818200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T18:02:02.403 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-26T18:02:02.403 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-26T18:02:02.403 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-03-26T18:02:02.403 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-26T18:02:02.403 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-03-26T18:02:02.403 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-26T18:02:02.403 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-26T18:02:02.403 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-26T18:02:02.403 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-03-26T18:02:02.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:02:02.404 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T18:02:02.421 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42081sec preferred_lft 42081sec 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-03-26T18:02:02.422 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-03-26T18:02:02.422 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-26T18:02:02.423 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T18:02:02.423 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-26T18:02:02.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2024-03-26T18:02:02.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2024-03-26T18:02:02.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2024-03-26T18:02:02.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-26T18:02:02.423 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-26T18:02:02.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:02:02.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:02:02.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:02:02.595 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-26T18:02:02.624 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:02 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:02.372+0000 7f353b818200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T18:02:02.626 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-03-26T18:02:02.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:02:02.628 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-03-26T18:02:02.684 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-03-26T18:02:02.684 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-26T18:02:02.684 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-26T18:02:02.684 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-26T18:02:02.685 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-26T18:02:02.685 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T18:02:02.685 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-26T18:02:02.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-26T18:02:02.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-26T18:02:02.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-26T18:02:02.685 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-26T18:02:02.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:02:02.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:02:02.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:02:02.834 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-03-26T18:02:02.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:02:02.850 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-03-26T18:02:02.855 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:02.433+0000 7f6c9342c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T18:02:02.855 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:02.604+0000 7f6c9342c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T18:02:02.905 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T18:02:02.905 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-26T18:02:02.905 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--tmp-cephfs 2024-03-26T18:02:02.905 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-03-26T18:02:02.905 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-26T18:02:02.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:02:03.001 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:02 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:02.738+0000 7f353b818200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T18:02:03.001 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:02 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:02.835+0000 7f353b818200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T18:02:03.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:02:03.037 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-03-26T18:02:03.037 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T18:02:03.037 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-26T18:02:03.037 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-03-26T18:02:03.038 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-26T18:02:03.038 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-03-26T18:02:03.038 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-03-26T18:02:03.038 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-03-26T18:02:03.038 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-26T18:02:03.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:02:03.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:02:03.186 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T18:02:03.186 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-03-26T18:02:03.186 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-03-26T18:02:03.186 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2024-03-26T18:02:03.186 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-03-26T18:02:03.233 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:02.947+0000 7f6c9342c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T18:02:03.233 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:03.038+0000 7f6c9342c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T18:02:03.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:02.781+0000 7fe33c897200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T18:02:03.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:02.962+0000 7fe33c897200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T18:02:03.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:02:03.270 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:03 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:03.049+0000 7f353b818200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:02:03.270 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:03 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:03.258+0000 7f353b818200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T18:02:03.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:02:03.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T18:02:03.281 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /tmp/cephfs 2024-03-26T18:02:03.308 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/tmp/cephfs' 2024-03-26T18:02:03.309 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T18:02:03.309 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /tmp/cephfs 2024-03-26T18:02:03.364 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-26T18:02:03.364 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-26T18:02:03.364 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/af486460-0fa8-40db-9e10-2412dd703b0d 2024-03-26T18:02:03.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:02:03.365 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/af486460-0fa8-40db-9e10-2412dd703b0d /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-03-26T18:02:03.454 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-03-26T18:02:03.454 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-03-26T18:02:03.588 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-03-26T18:02:03.588 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-03-26T18:02:03.588 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.115:3300,172.21.15.171:3300" 2024-03-26T18:02:03.588 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 0@5a3632a4-eb99-11ee-95d0-87774f69a715.cephfs=/volumes/qa/sv_1/af486460-0fa8-40db-9e10-2412dd703b0d 2024-03-26T18:02:03.588 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.49:3300/172.21.15.115:3300/172.21.15.171:3300" will pass to kernel 2024-03-26T18:02:03.590 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-26T18:02:03.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:02:03.591 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /tmp/cephfs 2024-03-26T18:02:03.605 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:03.252+0000 7f6c9342c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:02:03.605 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:03.441+0000 7f6c9342c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T18:02:03.633 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T18:02:03.681 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:03.369+0000 7fe33c897200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T18:02:03.682 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:03.473+0000 7fe33c897200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T18:02:03.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-26T18:02:03.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:02:03.735 INFO:teuthology.orchestra.run.smithi049.stdout:client25436 2024-03-26T18:02:03.735 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T18:02:03.735 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T18:02:03.752 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T18:02:03.826 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:03 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:03.569+0000 7f353b818200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:02:03.827 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:03 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:03.844+0000 7f353b818200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T18:02:03.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-26T18:02:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:02:03.850 INFO:teuthology.orchestra.run.smithi049.stdout:client25436 2024-03-26T18:02:03.851 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T18:02:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-26T18:02:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:02:03.950 INFO:teuthology.orchestra.run.smithi049.stdout:client25436 2024-03-26T18:02:03.950 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715.client25436/status') 2024-03-26T18:02:03.980 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:03.734+0000 7f6c9342c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:02:03.986 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:03.701+0000 7fe33c897200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:02:03.986 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:03.942+0000 7fe33c897200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T18:02:04.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T18:02:04.016 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.25436 (3)172.21.15.49:0/1801013281 2024-03-26T18:02:04.016 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2024-03-26T18:02:04.017 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2024-03-26T18:02:04.017 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2024-03-26T18:02:04.017 INFO:teuthology.orchestra.run.smithi049.stderr:71 bytes copied, 2.3254e-05 s, 3.1 MB/s 2024-03-26T18:02:04.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T18:02:04.024 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T18:02:04.033 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T18:02:04.033 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T18:02:04.034 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-26T18:02:04.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.162+0000 7f031fb6f640 1 -- 172.21.15.49:0/2701786242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03181034c0 msgr2=0x7f0318109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:04.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.162+0000 7f031fb6f640 1 --2- 172.21.15.49:0/2701786242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03181034c0 0x7f0318109d50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0310009ca0 tx=0x7f031002c0f0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.163+0000 7f031fb6f640 1 -- 172.21.15.49:0/2701786242 shutdown_connections 2024-03-26T18:02:04.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.163+0000 7f031fb6f640 1 --2- 172.21.15.49:0/2701786242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0318102b00 0x7f0318102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.163+0000 7f031fb6f640 1 --2- 172.21.15.49:0/2701786242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03181034c0 0x7f0318109d50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.163+0000 7f031fb6f640 1 --2- 172.21.15.49:0/2701786242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0318101910 0x7f0318101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.163+0000 7f031fb6f640 1 -- 172.21.15.49:0/2701786242 >> 172.21.15.49:0/2701786242 conn(0x7f03180fce80 msgr2=0x7f03180ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:04.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.163+0000 7f031fb6f640 1 -- 172.21.15.49:0/2701786242 shutdown_connections 2024-03-26T18:02:04.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.163+0000 7f031fb6f640 1 -- 172.21.15.49:0/2701786242 wait complete. 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031fb6f640 1 Processor -- start 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031fb6f640 1 -- start start 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031fb6f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318101910 0x7f0318197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031fb6f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0318102b00 0x7f0318198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031fb6f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03181034c0 0x7f031819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031fb6f640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f031810d590 con 0x7f0318101910 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031fb6f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f031810d410 con 0x7f03181034c0 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031fb6f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f031810d710 con 0x7f0318102b00 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031d90b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318101910 0x7f0318197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031d90b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318101910 0x7f0318197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52922/0 (socket says 172.21.15.49:52922) 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031d90b640 1 -- 172.21.15.49:0/3338130577 learned_addr learned my addr 172.21.15.49:0/3338130577 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:02:04.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031d10a640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0318102b00 0x7f0318198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031e10c640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03181034c0 0x7f031819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031d90b640 1 -- 172.21.15.49:0/3338130577 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0318102b00 msgr2=0x7f0318198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031d90b640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0318102b00 0x7f0318198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031d90b640 1 -- 172.21.15.49:0/3338130577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03181034c0 msgr2=0x7f031819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031d90b640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03181034c0 0x7f031819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.164+0000 7f031d90b640 1 -- 172.21.15.49:0/3338130577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031819f730 con 0x7f0318101910 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.165+0000 7f031d90b640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318101910 0x7f0318197b10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f031400e9c0 tx=0x7f031400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.165+0000 7f030effd640 1 -- 172.21.15.49:0/3338130577 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031400cd80 con 0x7f0318101910 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.165+0000 7f031fb6f640 1 -- 172.21.15.49:0/3338130577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031819fa20 con 0x7f0318101910 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.165+0000 7f030effd640 1 -- 172.21.15.49:0/3338130577 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03140035f0 con 0x7f0318101910 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.165+0000 7f030effd640 1 -- 172.21.15.49:0/3338130577 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314010690 con 0x7f0318101910 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.165+0000 7f031fb6f640 1 -- 172.21.15.49:0/3338130577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031819ff70 con 0x7f0318101910 2024-03-26T18:02:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.166+0000 7f031fb6f640 1 -- 172.21.15.49:0/3338130577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0318108bd0 con 0x7f0318101910 2024-03-26T18:02:04.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.167+0000 7f030effd640 1 -- 172.21.15.49:0/3338130577 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 19) ==== 137871+0+0 (secure 0 0 0) 0x7f03140037a0 con 0x7f0318101910 2024-03-26T18:02:04.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.168+0000 7f030effd640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03040a3010 0x7f03040a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:04.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.168+0000 7f031d10a640 1 -- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03040a3010 msgr2=0x7f03040a54d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/3127469495 2024-03-26T18:02:04.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.168+0000 7f030effd640 1 -- 172.21.15.49:0/3338130577 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f03140cfc60 con 0x7f0318101910 2024-03-26T18:02:04.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.168+0000 7f031d10a640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03040a3010 0x7f03040a54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:02:04.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.171+0000 7f030effd640 1 -- 172.21.15.49:0/3338130577 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f031409c7a0 con 0x7f0318101910 2024-03-26T18:02:04.289 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:03.998+0000 7f6c9342c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T18:02:04.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.369+0000 7f031d10a640 1 -- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03040a3010 msgr2=0x7f03040a54d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/3127469495 2024-03-26T18:02:04.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.369+0000 7f031d10a640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03040a3010 0x7f03040a54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T18:02:04.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.396+0000 7f031fb6f640 1 -- 172.21.15.49:0/3338130577 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f031802d400 con 0x7f0318101910 2024-03-26T18:02:04.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.448+0000 7f030effd640 1 -- 172.21.15.49:0/3338130577 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f031409c0a0 con 0x7f0318101910 2024-03-26T18:02:04.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 -- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03040a3010 msgr2=0x7f03040a54d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:02:04.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03040a3010 0x7f03040a54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 -- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318101910 msgr2=0x7f0318197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:04.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318101910 0x7f0318197b10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f031400e9c0 tx=0x7f031400ee80 comp rx=0 tx=0).stop 2024-03-26T18:02:04.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 -- 172.21.15.49:0/3338130577 shutdown_connections 2024-03-26T18:02:04.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f03040a3010 0x7f03040a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0318102b00 0x7f0318198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03181034c0 0x7f031819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 --2- 172.21.15.49:0/3338130577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0318101910 0x7f0318197b10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 -- 172.21.15.49:0/3338130577 >> 172.21.15.49:0/3338130577 conn(0x7f03180fce80 msgr2=0x7f0318105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:04.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 -- 172.21.15.49:0/3338130577 shutdown_connections 2024-03-26T18:02:04.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.451+0000 7f031fb6f640 1 -- 172.21.15.49:0/3338130577 wait complete. 2024-03-26T18:02:04.435 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:04 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:04.101+0000 7f353b818200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T18:02:04.445 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-26T18:02:04.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:04 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3338130577' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T18:02:04.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:04 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:04.254+0000 7fe33c897200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T18:02:04.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.576+0000 7f983e13b640 1 -- 172.21.15.49:0/3977779762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9838104cf0 msgr2=0x7f98381070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:04.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.576+0000 7f983e13b640 1 --2- 172.21.15.49:0/3977779762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9838104cf0 0x7f98381070e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9828009900 tx=0x7f982802bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:04.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.576+0000 7f983e13b640 1 -- 172.21.15.49:0/3977779762 shutdown_connections 2024-03-26T18:02:04.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.576+0000 7f983e13b640 1 --2- 172.21.15.49:0/3977779762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9838107620 0x7f9838109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.576+0000 7f983e13b640 1 --2- 172.21.15.49:0/3977779762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9838104cf0 0x7f98381070e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.576+0000 7f983e13b640 1 --2- 172.21.15.49:0/3977779762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98381023c0 0x7f98381047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.576+0000 7f983e13b640 1 -- 172.21.15.49:0/3977779762 >> 172.21.15.49:0/3977779762 conn(0x7f98380fbd50 msgr2=0x7f98380fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.576+0000 7f983e13b640 1 -- 172.21.15.49:0/3977779762 shutdown_connections 2024-03-26T18:02:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.576+0000 7f983e13b640 1 -- 172.21.15.49:0/3977779762 wait complete. 2024-03-26T18:02:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f983e13b640 1 Processor -- start 2024-03-26T18:02:04.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f983e13b640 1 -- start start 2024-03-26T18:02:04.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f983e13b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98381023c0 0x7f9838197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:04.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f983e13b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9838104cf0 0x7f98381980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:04.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f983e13b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9838107620 0x7f983819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:04.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f983e13b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f983810d550 con 0x7f9838107620 2024-03-26T18:02:04.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f983e13b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f983810d3d0 con 0x7f98381023c0 2024-03-26T18:02:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f983e13b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f983810d6d0 con 0x7f9838104cf0 2024-03-26T18:02:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f98377fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98381023c0 0x7f9838197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f9837fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9838107620 0x7f983819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f9836ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9838104cf0 0x7f98381980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f98377fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98381023c0 0x7f9838197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:34846/0 (socket says 172.21.15.49:34846) 2024-03-26T18:02:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.577+0000 7f98377fe640 1 -- 172.21.15.49:0/3429976846 learned_addr learned my addr 172.21.15.49:0/3429976846 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:02:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f9836ffd640 1 -- 172.21.15.49:0/3429976846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98381023c0 msgr2=0x7f9838197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:04.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f9836ffd640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98381023c0 0x7f9838197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f9836ffd640 1 -- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9838107620 msgr2=0x7f983819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f9836ffd640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9838107620 0x7f983819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f9836ffd640 1 -- 172.21.15.49:0/3429976846 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983819f7b0 con 0x7f9838104cf0 2024-03-26T18:02:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f98377fe640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98381023c0 0x7f9838197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f9837fff640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9838107620 0x7f983819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f9836ffd640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9838104cf0 0x7f98381980d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f982802c170 tx=0x7f982802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:04.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828041070 con 0x7f9838104cf0 2024-03-26T18:02:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f983e13b640 1 -- 172.21.15.49:0/3429976846 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983819fa40 con 0x7f9838104cf0 2024-03-26T18:02:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9828004450 con 0x7f9838104cf0 2024-03-26T18:02:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828003e50 con 0x7f9838104cf0 2024-03-26T18:02:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.578+0000 7f983e13b640 1 -- 172.21.15.49:0/3429976846 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f983819ff00 con 0x7f9838104cf0 2024-03-26T18:02:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.579+0000 7f983e13b640 1 -- 172.21.15.49:0/3429976846 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98380fd840 con 0x7f9838104cf0 2024-03-26T18:02:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.580+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 19) ==== 137871+0+0 (secure 0 0 0) 0x7f9828044070 con 0x7f9838104cf0 2024-03-26T18:02:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.581+0000 7f9834ff9640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f98240a31b0 0x7f98240a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:04.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.581+0000 7f98377fe640 1 -- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f98240a31b0 msgr2=0x7f98240a5670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/3127469495 2024-03-26T18:02:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.581+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9129+0+0 (secure 0 0 0) 0x7f98280f13f0 con 0x7f9838104cf0 2024-03-26T18:02:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.581+0000 7f98377fe640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f98240a31b0 0x7f98240a5670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T18:02:04.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.584+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f98280bd850 con 0x7f9838104cf0 2024-03-26T18:02:04.605 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:04 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3338130577' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T18:02:04.606 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:04.308+0000 7f6c9342c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T18:02:04.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:04 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3338130577' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T18:02:04.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.782+0000 7f98377fe640 1 -- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f98240a31b0 msgr2=0x7f98240a5670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/3127469495 2024-03-26T18:02:04.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.782+0000 7f98377fe640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f98240a31b0 0x7f98240a5670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T18:02:04.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:04.799+0000 7f983e13b640 1 -- 172.21.15.49:0/3429976846 --> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f9838061e60 con 0x7f98240a31b0 2024-03-26T18:02:04.862 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:04 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:04.560+0000 7fe33c897200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T18:02:04.862 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:04 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:04.881+0000 7fe33c897200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T18:02:05.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:05.183+0000 7f98377fe640 1 -- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f98240a31b0 msgr2=0x7f98240a5670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6800/3127469495 2024-03-26T18:02:05.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:05.183+0000 7f98377fe640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f98240a31b0 0x7f98240a5670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T18:02:05.251 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:04 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:04.968+0000 7f353b818200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T18:02:05.436 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:05 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:05.151+0000 7f6c9342c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T18:02:05.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:05 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3338130577' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T18:02:05.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:05 smithi115 ceph-mon[36174]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:05.736 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:05 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3338130577' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T18:02:05.737 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:05 smithi171 ceph-mon[36033]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:05.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:05 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3338130577' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T18:02:05.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:05 smithi049 ceph-mon[32331]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:05.751 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:05 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:05.568+0000 7f353b818200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T18:02:05.751 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:05 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:05.672+0000 7f353b818200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T18:02:05.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:05.937+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mgrmap(e 20) ==== 137539+0+0 (secure 0 0 0) 0x7f98280035f0 con 0x7f9838104cf0 2024-03-26T18:02:05.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:05.938+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f98240a31b0 msgr2=0x7f98240a5670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T18:02:05.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:05.938+0000 7f9834ff9640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/3127469495,v1:172.21.15.49:6801/3127469495] conn(0x7f98240a31b0 0x7f98240a5670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.093 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:05 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:05.758+0000 7fe33c897200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T18:02:06.093 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:05 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:05.755+0000 7f6c9342c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T18:02:06.093 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:05 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:05.903+0000 7f6c9342c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T18:02:06.163 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:05 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:05.904+0000 7f353b818200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T18:02:06.355 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z[37780]: 2024-03-26T18:02:06.166+0000 7f6c9342c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T18:02:06.438 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.181+0000 7f349d9ac700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.439 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.181+0000 7f349d9ac700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.439 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.181+0000 7f349d9ac700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.439 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.181+0000 7f349d9ac700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.439 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.181+0000 7f349d9ac700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.439 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.181+0000 7f349a1a5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.439 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.181+0000 7f349a1a5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.439 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.181+0000 7f349a1a5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.439 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.181+0000 7f349a1a5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.439 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.181+0000 7f349a1a5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: Active manager daemon x restarted 2024-03-26T18:02:06.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: Activating manager daemon x 2024-03-26T18:02:06.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T18:02:06.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: mgrmap e20: x(active, starting, since 0.0147214s), standbys: z, y 2024-03-26T18:02:06.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:02:06.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:02:06.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:02:06.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T18:02:06.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T18:02:06.693 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T18:02:06.694 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T18:02:06.695 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: Manager daemon x is now available 2024-03-26T18:02:06.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:02:06.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T18:02:06.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T18:02:06.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: Standby manager daemon z restarted 2024-03-26T18:02:06.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: Standby manager daemon z started 2024-03-26T18:02:06.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T18:02:06.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:02:06.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T18:02:06.696 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:02:06.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: Active manager daemon x restarted 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: Activating manager daemon x 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: mgrmap e20: x(active, starting, since 0.0147214s), standbys: z, y 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:02:06.737 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:02:06.738 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:02:06.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:02:06.739 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:06.459+0000 7fe33c897200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T18:02:06.739 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:06.624+0000 7fe33c897200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T18:02:06.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:02:06.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:02:06.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:02:06.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:02:06.739 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: Manager daemon x is now available 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: Standby manager daemon z restarted 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: Standby manager daemon z started 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:02:06.740 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T18:02:06.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:02:06.855 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: Active manager daemon x restarted 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: Activating manager daemon x 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: mgrmap e20: x(active, starting, since 0.0147214s), standbys: z, y 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:02:06.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T18:02:06.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T18:02:06.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T18:02:06.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T18:02:06.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T18:02:06.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T18:02:06.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T18:02:06.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T18:02:06.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T18:02:06.858 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: Manager daemon x is now available 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: Standby manager daemon z restarted 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: Standby manager daemon z started 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T18:02:06.859 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:06 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.171:0/1160927288' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:02:06.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.939+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 <== mon.1 v2:172.21.15.171:3300/0 8 ==== mgrmap(e 21) ==== 137666+0+0 (secure 0 0 0) 0x7f98280b9ec0 con 0x7f9838104cf0 2024-03-26T18:02:06.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.940+0000 7f9834ff9640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f9824051f10 0x7f98240522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.940+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 --> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f9824044ff0 con 0x7f9824051f10 2024-03-26T18:02:06.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.944+0000 7f98377fe640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f9824051f10 0x7f98240522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.947+0000 7f98377fe640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f9824051f10 0x7f98240522f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f982000a010 tx=0x7f9820005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:06.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.974+0000 7f9834ff9640 1 -- 172.21.15.49:0/3429976846 <== mgr.15516 v2:172.21.15.49:6800/2879593470 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f9824044ff0 con 0x7f9824051f10 2024-03-26T18:02:06.955 INFO:teuthology.orchestra.run.smithi049.stdout:{} 2024-03-26T18:02:06.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.977+0000 7f983e13b640 1 -- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f9824051f10 msgr2=0x7f98240522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.977+0000 7f983e13b640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f9824051f10 0x7f98240522f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f982000a010 tx=0x7f9820005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:06.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.977+0000 7f983e13b640 1 -- 172.21.15.49:0/3429976846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9838104cf0 msgr2=0x7f98381980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.977+0000 7f983e13b640 1 --2- 172.21.15.49:0/3429976846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9838104cf0 0x7f98381980d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f982802c170 tx=0x7f982802c650 comp rx=0 tx=0).stop 2024-03-26T18:02:06.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.977+0000 7f9837fff640 1 -- 172.21.15.49:0/3429976846 reap_dead start 2024-03-26T18:02:06.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.977+0000 7f983e13b640 1 -- 172.21.15.49:0/3429976846 shutdown_connections 2024-03-26T18:02:06.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.977+0000 7f983e13b640 1 -- 172.21.15.49:0/3429976846 >> 172.21.15.49:0/3429976846 conn(0x7f98380fbd50 msgr2=0x7f9838102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:06.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.977+0000 7f983e13b640 1 -- 172.21.15.49:0/3429976846 shutdown_connections 2024-03-26T18:02:06.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:06.977+0000 7f983e13b640 1 -- 172.21.15.49:0/3429976846 wait complete. 2024-03-26T18:02:06.972 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-26T18:02:06.984 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T18:02:06.993 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T18:02:06.993 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T18:02:06.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-26T18:02:06.993 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.955+0000 7f34981a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.993 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.955+0000 7f34981a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.994 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.955+0000 7f34981a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.994 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.955+0000 7f34981a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:06.994 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:02:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:02:06.955+0000 7f34981a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.129+0000 7fd05d562640 1 -- 172.21.15.49:0/4288110657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd058107660 msgr2=0x7fd058109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.129+0000 7fd05d562640 1 --2- 172.21.15.49:0/4288110657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd058107660 0x7fd058109a50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd050009ca0 tx=0x7fd05002c0f0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.129+0000 7fd05d562640 1 -- 172.21.15.49:0/4288110657 shutdown_connections 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 --2- 172.21.15.49:0/4288110657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd058104d30 0x7fd058107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 --2- 172.21.15.49:0/4288110657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd058107660 0x7fd058109a50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 --2- 172.21.15.49:0/4288110657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd058102400 0x7fd0581047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 -- 172.21.15.49:0/4288110657 >> 172.21.15.49:0/4288110657 conn(0x7fd0580fbd50 msgr2=0x7fd0580fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 -- 172.21.15.49:0/4288110657 shutdown_connections 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 -- 172.21.15.49:0/4288110657 wait complete. 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 Processor -- start 2024-03-26T18:02:07.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 -- start start 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd058102400 0x7fd058197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd058104d30 0x7fd0581980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd058107660 0x7fd05819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd05810d580 con 0x7fd058104d30 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd05810d400 con 0x7fd058102400 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.130+0000 7fd05d562640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd05810d700 con 0x7fd058107660 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0567fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd058104d30 0x7fd0581980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd056ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd058102400 0x7fd058197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0567fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd058104d30 0x7fd0581980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53098/0 (socket says 172.21.15.49:53098) 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0567fc640 1 -- 172.21.15.49:0/3222847814 learned_addr learned my addr 172.21.15.49:0/3222847814 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0577fe640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd058107660 0x7fd05819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0567fc640 1 -- 172.21.15.49:0/3222847814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd058107660 msgr2=0x7fd05819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0567fc640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd058107660 0x7fd05819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0567fc640 1 -- 172.21.15.49:0/3222847814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd058102400 msgr2=0x7fd058197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0567fc640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd058102400 0x7fd058197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0567fc640 1 -- 172.21.15.49:0/3222847814 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05819f790 con 0x7fd058104d30 2024-03-26T18:02:07.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0577fe640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd058107660 0x7fd05819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:07.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd0567fc640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd058104d30 0x7fd0581980b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00e9f0 tx=0x7fd04c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:07.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd03ffff640 1 -- 172.21.15.49:0/3222847814 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00ccc0 con 0x7fd058104d30 2024-03-26T18:02:07.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd03ffff640 1 -- 172.21.15.49:0/3222847814 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd04c005500 con 0x7fd058104d30 2024-03-26T18:02:07.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd05d562640 1 -- 172.21.15.49:0/3222847814 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05819fa80 con 0x7fd058104d30 2024-03-26T18:02:07.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd03ffff640 1 -- 172.21.15.49:0/3222847814 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c010690 con 0x7fd058104d30 2024-03-26T18:02:07.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.131+0000 7fd05d562640 1 -- 172.21.15.49:0/3222847814 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05819ff80 con 0x7fd058104d30 2024-03-26T18:02:07.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.133+0000 7fd05d562640 1 -- 172.21.15.49:0/3222847814 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0580fd840 con 0x7fd058104d30 2024-03-26T18:02:07.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.134+0000 7fd03ffff640 1 -- 172.21.15.49:0/3222847814 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 21) ==== 137666+0+0 (secure 0 0 0) 0x7fd04c0046c0 con 0x7fd058104d30 2024-03-26T18:02:07.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.135+0000 7fd03ffff640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd0400a2dd0 0x7fd0400a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.135+0000 7fd03ffff640 1 -- 172.21.15.49:0/3222847814 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9387+0+0 (secure 0 0 0) 0x7fd04c0cf7c0 con 0x7fd058104d30 2024-03-26T18:02:07.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.135+0000 7fd056ffd640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd0400a2dd0 0x7fd0400a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.136+0000 7fd056ffd640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd0400a2dd0 0x7fd0400a5290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd048009bd0 tx=0x7fd048005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:07.125 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.144+0000 7fd03ffff640 1 -- 172.21.15.49:0/3222847814 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd04c09c460 con 0x7fd058104d30 2024-03-26T18:02:07.236 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:02:06 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y[37556]: 2024-03-26T18:02:06.962+0000 7fe33c897200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T18:02:07.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.354+0000 7fd05d562640 1 -- 172.21.15.49:0/3222847814 --> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd058061d00 con 0x7fd0400a2dd0 2024-03-26T18:02:07.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.428+0000 7fd03ffff640 1 -- 172.21.15.49:0/3222847814 <== mgr.15516 v2:172.21.15.49:6800/2879593470 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd058061d00 con 0x7fd0400a2dd0 2024-03-26T18:02:07.409 INFO:teuthology.orchestra.run.smithi049.stdout:Schedule set for path / 2024-03-26T18:02:07.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 -- 172.21.15.49:0/3222847814 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd0400a2dd0 msgr2=0x7fd0400a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd0400a2dd0 0x7fd0400a5290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd048009bd0 tx=0x7fd048005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 -- 172.21.15.49:0/3222847814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd058104d30 msgr2=0x7fd0581980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd058104d30 0x7fd0581980b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00e9f0 tx=0x7fd04c00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 -- 172.21.15.49:0/3222847814 shutdown_connections 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd0400a2dd0 0x7fd0400a5290 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd058107660 0x7fd05819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd058104d30 0x7fd0581980b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 --2- 172.21.15.49:0/3222847814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd058102400 0x7fd058197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 -- 172.21.15.49:0/3222847814 >> 172.21.15.49:0/3222847814 conn(0x7fd0580fbd50 msgr2=0x7fd058102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 -- 172.21.15.49:0/3222847814 shutdown_connections 2024-03-26T18:02:07.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.430+0000 7fd05d562640 1 -- 172.21.15.49:0/3222847814 wait complete. 2024-03-26T18:02:07.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-26T18:02:07.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 -- 172.21.15.49:0/2708305005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd6c100aa0 msgr2=0x7fcd6c100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 --2- 172.21.15.49:0/2708305005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd6c100aa0 0x7fcd6c100f20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009930 tx=0x7fcd6002bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:07.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 -- 172.21.15.49:0/2708305005 shutdown_connections 2024-03-26T18:02:07.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 --2- 172.21.15.49:0/2708305005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd6c101460 0x7fcd6c108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 --2- 172.21.15.49:0/2708305005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd6c100aa0 0x7fcd6c100f20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 --2- 172.21.15.49:0/2708305005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd6c101e00 0x7fcd6c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 -- 172.21.15.49:0/2708305005 >> 172.21.15.49:0/2708305005 conn(0x7fcd6c0fb510 msgr2=0x7fcd6c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 -- 172.21.15.49:0/2708305005 shutdown_connections 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 -- 172.21.15.49:0/2708305005 wait complete. 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 Processor -- start 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.561+0000 7fcd71f3b640 1 -- start start 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd71f3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd6c100aa0 0x7fcd6c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd71f3b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd6c101460 0x7fcd6c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd71f3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd6c101e00 0x7fcd6c1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd71f3b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c10c4c0 con 0x7fcd6c101e00 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd71f3b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c10c340 con 0x7fcd6c100aa0 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd71f3b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c10c640 con 0x7fcd6c101460 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd6c101e00 0x7fcd6c1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6affd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd6c101460 0x7fcd6c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6affd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd6c101460 0x7fcd6c19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:39404/0 (socket says 172.21.15.49:39404) 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd6c101e00 0x7fcd6c1a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53112/0 (socket says 172.21.15.49:53112) 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd6c100aa0 0x7fcd6c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6affd640 1 -- 172.21.15.49:0/1799982590 learned_addr learned my addr 172.21.15.49:0/1799982590 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6affd640 1 -- 172.21.15.49:0/1799982590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd6c100aa0 msgr2=0x7fcd6c199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6affd640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd6c100aa0 0x7fcd6c199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6affd640 1 -- 172.21.15.49:0/1799982590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd6c101e00 msgr2=0x7fcd6c1a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6affd640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd6c101e00 0x7fcd6c1a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6affd640 1 -- 172.21.15.49:0/1799982590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6c1a1a80 con 0x7fcd6c101460 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.562+0000 7fcd6affd640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd6c101460 0x7fcd6c19a3a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd6002efd0 tx=0x7fcd6002c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.563+0000 7fcd68ff9640 1 -- 172.21.15.49:0/1799982590 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60041070 con 0x7fcd6c101460 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.563+0000 7fcd68ff9640 1 -- 172.21.15.49:0/1799982590 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd60004450 con 0x7fcd6c101460 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.563+0000 7fcd68ff9640 1 -- 172.21.15.49:0/1799982590 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60034440 con 0x7fcd6c101460 2024-03-26T18:02:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.563+0000 7fcd71f3b640 1 -- 172.21.15.49:0/1799982590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6c1a1d10 con 0x7fcd6c101460 2024-03-26T18:02:07.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.563+0000 7fcd71f3b640 1 -- 172.21.15.49:0/1799982590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6c1a21b0 con 0x7fcd6c101460 2024-03-26T18:02:07.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.564+0000 7fcd71f3b640 1 -- 172.21.15.49:0/1799982590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6c1a2380 con 0x7fcd6c101460 2024-03-26T18:02:07.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.566+0000 7fcd68ff9640 1 -- 172.21.15.49:0/1799982590 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 21) ==== 137666+0+0 (secure 0 0 0) 0x7fcd60044070 con 0x7fcd6c101460 2024-03-26T18:02:07.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.566+0000 7fcd68ff9640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fcd540a2dd0 0x7fcd540a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.566+0000 7fcd6b7fe640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fcd540a2dd0 0x7fcd540a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.566+0000 7fcd68ff9640 1 -- 172.21.15.49:0/1799982590 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9387+0+0 (secure 0 0 0) 0x7fcd600f12b0 con 0x7fcd6c101460 2024-03-26T18:02:07.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.567+0000 7fcd6b7fe640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fcd540a2dd0 0x7fcd540a5290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c069ea0 tx=0x7fcd5c005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:07.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.571+0000 7fcd68ff9640 1 -- 172.21.15.49:0/1799982590 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcd600bd900 con 0x7fcd6c101460 2024-03-26T18:02:07.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.788+0000 7fcd71f3b640 1 -- 172.21.15.49:0/1799982590 --> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fcd6c0fd660 con 0x7fcd540a2dd0 2024-03-26T18:02:07.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.851+0000 7fcd68ff9640 1 -- 172.21.15.49:0/1799982590 <== mgr.15516 v2:172.21.15.49:6800/2879593470 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fcd6c0fd660 con 0x7fcd540a2dd0 2024-03-26T18:02:07.833 INFO:teuthology.orchestra.run.smithi049.stdout:Retention added to path / 2024-03-26T18:02:07.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.853+0000 7fcd71f3b640 1 -- 172.21.15.49:0/1799982590 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fcd540a2dd0 msgr2=0x7fcd540a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.853+0000 7fcd71f3b640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fcd540a2dd0 0x7fcd540a5290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c069ea0 tx=0x7fcd5c005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:07.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.853+0000 7fcd71f3b640 1 -- 172.21.15.49:0/1799982590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd6c101460 msgr2=0x7fcd6c19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.853+0000 7fcd71f3b640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd6c101460 0x7fcd6c19a3a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd6002efd0 tx=0x7fcd6002c680 comp rx=0 tx=0).stop 2024-03-26T18:02:07.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.854+0000 7fcd71f3b640 1 -- 172.21.15.49:0/1799982590 shutdown_connections 2024-03-26T18:02:07.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.854+0000 7fcd71f3b640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fcd540a2dd0 0x7fcd540a5290 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.854+0000 7fcd71f3b640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd6c101460 0x7fcd6c19a3a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.854+0000 7fcd71f3b640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd6c101e00 0x7fcd6c1a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.854+0000 7fcd71f3b640 1 --2- 172.21.15.49:0/1799982590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd6c100aa0 0x7fcd6c199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.854+0000 7fcd71f3b640 1 -- 172.21.15.49:0/1799982590 >> 172.21.15.49:0/1799982590 conn(0x7fcd6c0fb510 msgr2=0x7fcd6c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:07.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.854+0000 7fcd71f3b640 1 -- 172.21.15.49:0/1799982590 shutdown_connections 2024-03-26T18:02:07.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.854+0000 7fcd71f3b640 1 -- 172.21.15.49:0/1799982590 wait complete. 2024-03-26T18:02:07.847 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-26T18:02:07.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.987+0000 7fa7b4f68640 1 -- 172.21.15.49:0/3516657588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b0100820 msgr2=0x7fa7b0100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.987+0000 7fa7b4f68640 1 --2- 172.21.15.49:0/3516657588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b0100820 0x7fa7b0100c40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009900 tx=0x7fa7a002bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.987+0000 7fa7b4f68640 1 -- 172.21.15.49:0/3516657588 shutdown_connections 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.987+0000 7fa7b4f68640 1 --2- 172.21.15.49:0/3516657588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7b0101a10 0x7fa7b0101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.987+0000 7fa7b4f68640 1 --2- 172.21.15.49:0/3516657588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b01023d0 0x7fa7b0108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.987+0000 7fa7b4f68640 1 --2- 172.21.15.49:0/3516657588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b0100820 0x7fa7b0100c40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.987+0000 7fa7b4f68640 1 -- 172.21.15.49:0/3516657588 >> 172.21.15.49:0/3516657588 conn(0x7fa7b00fbd90 msgr2=0x7fa7b00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.987+0000 7fa7b4f68640 1 -- 172.21.15.49:0/3516657588 shutdown_connections 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.987+0000 7fa7b4f68640 1 -- 172.21.15.49:0/3516657588 wait complete. 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7b4f68640 1 Processor -- start 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7b4f68640 1 -- start start 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7b4f68640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7b0100820 0x7fa7b006a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7b4f68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b0101a10 0x7fa7b019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7b4f68640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b01023d0 0x7fa7b01a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7b4f68640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010c470 con 0x7fa7b0101a10 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7b4f68640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010c2f0 con 0x7fa7b01023d0 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7b4f68640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010c5f0 con 0x7fa7b0100820 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7add9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b0101a10 0x7fa7b019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7add9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b0101a10 0x7fa7b019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53136/0 (socket says 172.21.15.49:53136) 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7add9b640 1 -- 172.21.15.49:0/1210176723 learned_addr learned my addr 172.21.15.49:0/1210176723 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7ae59c640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7b0100820 0x7fa7b006a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.988+0000 7fa7aed9d640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b01023d0 0x7fa7b01a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa7add9b640 1 -- 172.21.15.49:0/1210176723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7b0100820 msgr2=0x7fa7b006a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa7add9b640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7b0100820 0x7fa7b006a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa7add9b640 1 -- 172.21.15.49:0/1210176723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b01023d0 msgr2=0x7fa7b01a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa7add9b640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b01023d0 0x7fa7b01a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa7add9b640 1 -- 172.21.15.49:0/1210176723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b01a1a30 con 0x7fa7b0101a10 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa7add9b640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b0101a10 0x7fa7b019ed90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400d950 tx=0x7fa7a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:07.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa79f7fe640 1 -- 172.21.15.49:0/1210176723 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4017030 con 0x7fa7b0101a10 2024-03-26T18:02:07.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa7b4f68640 1 -- 172.21.15.49:0/1210176723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b01a1d20 con 0x7fa7b0101a10 2024-03-26T18:02:07.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa79f7fe640 1 -- 172.21.15.49:0/1210176723 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a4004020 con 0x7fa7b0101a10 2024-03-26T18:02:07.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa79f7fe640 1 -- 172.21.15.49:0/1210176723 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40047a0 con 0x7fa7b0101a10 2024-03-26T18:02:07.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.989+0000 7fa7b4f68640 1 -- 172.21.15.49:0/1210176723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b01a2210 con 0x7fa7b0101a10 2024-03-26T18:02:07.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.990+0000 7fa7b4f68640 1 -- 172.21.15.49:0/1210176723 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b0107ae0 con 0x7fa7b0101a10 2024-03-26T18:02:07.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.992+0000 7fa79f7fe640 1 -- 172.21.15.49:0/1210176723 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 22) ==== 137666+0+0 (secure 0 0 0) 0x7fa7a40041d0 con 0x7fa7b0101a10 2024-03-26T18:02:07.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.993+0000 7fa79f7fe640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa7980a2e00 0x7fa7980a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:07.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.993+0000 7fa7ae59c640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa7980a2e00 0x7fa7980a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:07.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.993+0000 7fa79f7fe640 1 -- 172.21.15.49:0/1210176723 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9387+0+0 (secure 0 0 0) 0x7fa7a40cedf0 con 0x7fa7b0101a10 2024-03-26T18:02:07.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.993+0000 7fa7ae59c640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa7980a2e00 0x7fa7980a52c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0005ab0 tx=0x7fa7a0005fd0 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:07.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:07.998+0000 7fa79f7fe640 1 -- 172.21.15.49:0/1210176723 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa7a409ba90 con 0x7fa7b0101a10 2024-03-26T18:02:08.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.239+0000 7fa7b4f68640 1 -- 172.21.15.49:0/1210176723 --> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa7b0061d00 con 0x7fa7980a2e00 2024-03-26T18:02:08.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.247+0000 7fa79f7fe640 1 -- 172.21.15.49:0/1210176723 <== mgr.15516 v2:172.21.15.49:6800/2879593470 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fa7b0061d00 con 0x7fa7980a2e00 2024-03-26T18:02:08.228 INFO:teuthology.orchestra.run.smithi049.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-26T00:00:00", "created": "2024-03-26T18:02:07", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-26T18:02:08.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 -- 172.21.15.49:0/1210176723 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa7980a2e00 msgr2=0x7fa7980a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa7980a2e00 0x7fa7980a52c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0005ab0 tx=0x7fa7a0005fd0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 -- 172.21.15.49:0/1210176723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b0101a10 msgr2=0x7fa7b019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b0101a10 0x7fa7b019ed90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400d950 tx=0x7fa7a400de10 comp rx=0 tx=0).stop 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 -- 172.21.15.49:0/1210176723 shutdown_connections 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa7980a2e00 0x7fa7980a52c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b01023d0 0x7fa7b01a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b0101a10 0x7fa7b019ed90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 --2- 172.21.15.49:0/1210176723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa7b0100820 0x7fa7b006a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 -- 172.21.15.49:0/1210176723 >> 172.21.15.49:0/1210176723 conn(0x7fa7b00fbd90 msgr2=0x7fa7b0104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.249+0000 7fa7b4f68640 1 -- 172.21.15.49:0/1210176723 shutdown_connections 2024-03-26T18:02:08.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.250+0000 7fa7b4f68640 1 -- 172.21.15.49:0/1210176723 wait complete. 2024-03-26T18:02:08.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:07 smithi115 ceph-mon[36174]: mgrmap e21: x(active, since 1.01901s), standbys: y, z 2024-03-26T18:02:08.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:07 smithi115 ceph-mon[36174]: Standby manager daemon y restarted 2024-03-26T18:02:08.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:07 smithi115 ceph-mon[36174]: Standby manager daemon y started 2024-03-26T18:02:08.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:07 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T18:02:08.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:07 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:02:08.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:07 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T18:02:08.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:07 smithi115 ceph-mon[36174]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:02:08.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:07 smithi115 ceph-mon[36174]: [26/Mar/2024:18:02:07] ENGINE Bus STARTING 2024-03-26T18:02:08.241 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:07 smithi049 ceph-mon[32331]: mgrmap e21: x(active, since 1.01901s), standbys: y, z 2024-03-26T18:02:08.241 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:07 smithi049 ceph-mon[32331]: Standby manager daemon y restarted 2024-03-26T18:02:08.241 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:07 smithi049 ceph-mon[32331]: Standby manager daemon y started 2024-03-26T18:02:08.241 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:07 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T18:02:08.241 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:07 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:02:08.242 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:07 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T18:02:08.242 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:07 smithi049 ceph-mon[32331]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:02:08.242 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:07 smithi049 ceph-mon[32331]: [26/Mar/2024:18:02:07] ENGINE Bus STARTING 2024-03-26T18:02:08.243 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-26T18:02:08.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:07 smithi171 ceph-mon[36033]: mgrmap e21: x(active, since 1.01901s), standbys: y, z 2024-03-26T18:02:08.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:07 smithi171 ceph-mon[36033]: Standby manager daemon y restarted 2024-03-26T18:02:08.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:07 smithi171 ceph-mon[36033]: Standby manager daemon y started 2024-03-26T18:02:08.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:07 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T18:02:08.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:07 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T18:02:08.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:07 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T18:02:08.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:07 smithi171 ceph-mon[36033]: from='mgr.? 172.21.15.115:0/2757469558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T18:02:08.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:07 smithi171 ceph-mon[36033]: [26/Mar/2024:18:02:07] ENGINE Bus STARTING 2024-03-26T18:02:08.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 -- 172.21.15.49:0/3177977997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc38100b10 msgr2=0x7fdc38100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 --2- 172.21.15.49:0/3177977997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc38100b10 0x7fdc38100f70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009900 tx=0x7fdc2c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 -- 172.21.15.49:0/3177977997 shutdown_connections 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 --2- 172.21.15.49:0/3177977997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc381014b0 0x7fdc38101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 --2- 172.21.15.49:0/3177977997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc38100b10 0x7fdc38100f70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 --2- 172.21.15.49:0/3177977997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc381001b0 0x7fdc381005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 -- 172.21.15.49:0/3177977997 >> 172.21.15.49:0/3177977997 conn(0x7fdc380fbd50 msgr2=0x7fdc380fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 -- 172.21.15.49:0/3177977997 shutdown_connections 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 -- 172.21.15.49:0/3177977997 wait complete. 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 Processor -- start 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.375+0000 7fdc3e86c640 1 -- start start 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3e86c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc381001b0 0x7fdc38199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:08.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3e86c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc38100b10 0x7fdc3819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3e86c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc381014b0 0x7fdc381a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3e86c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3810f740 con 0x7fdc381001b0 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3e86c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3810f5c0 con 0x7fdc38100b10 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3e86c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3810f8c0 con 0x7fdc381014b0 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc37fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc381001b0 0x7fdc38199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3ce09640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc381014b0 0x7fdc381a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc37fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc381001b0 0x7fdc38199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53170/0 (socket says 172.21.15.49:53170) 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc37fff640 1 -- 172.21.15.49:0/182740929 learned_addr learned my addr 172.21.15.49:0/182740929 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc377fe640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc38100b10 0x7fdc3819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3ce09640 1 -- 172.21.15.49:0/182740929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc38100b10 msgr2=0x7fdc3819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3ce09640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc38100b10 0x7fdc3819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3ce09640 1 -- 172.21.15.49:0/182740929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc381001b0 msgr2=0x7fdc38199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3ce09640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc381001b0 0x7fdc38199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc3ce09640 1 -- 172.21.15.49:0/182740929 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc381a19e0 con 0x7fdc381014b0 2024-03-26T18:02:08.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.376+0000 7fdc37fff640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc381001b0 0x7fdc38199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:08.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.377+0000 7fdc3ce09640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc381014b0 0x7fdc381a1370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009fd0 tx=0x7fdc3000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:08.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.377+0000 7fdc357fa640 1 -- 172.21.15.49:0/182740929 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000ce20 con 0x7fdc381014b0 2024-03-26T18:02:08.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.377+0000 7fdc3e86c640 1 -- 172.21.15.49:0/182740929 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc381a1cd0 con 0x7fdc381014b0 2024-03-26T18:02:08.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.377+0000 7fdc357fa640 1 -- 172.21.15.49:0/182740929 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc30005020 con 0x7fdc381014b0 2024-03-26T18:02:08.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.377+0000 7fdc357fa640 1 -- 172.21.15.49:0/182740929 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30010720 con 0x7fdc381014b0 2024-03-26T18:02:08.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.377+0000 7fdc3e86c640 1 -- 172.21.15.49:0/182740929 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc381a21a0 con 0x7fdc381014b0 2024-03-26T18:02:08.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.378+0000 7fdc3e86c640 1 -- 172.21.15.49:0/182740929 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc3810af00 con 0x7fdc381014b0 2024-03-26T18:02:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.380+0000 7fdc357fa640 1 -- 172.21.15.49:0/182740929 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 22) ==== 137666+0+0 (secure 0 0 0) 0x7fdc30005540 con 0x7fdc381014b0 2024-03-26T18:02:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.380+0000 7fdc357fa640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fdc200a2dd0 0x7fdc200a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.380+0000 7fdc37fff640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fdc200a2dd0 0x7fdc200a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.380+0000 7fdc357fa640 1 -- 172.21.15.49:0/182740929 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9387+0+0 (secure 0 0 0) 0x7fdc300cfc80 con 0x7fdc381014b0 2024-03-26T18:02:08.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.381+0000 7fdc37fff640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fdc200a2dd0 0x7fdc200a5290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc28011500 tx=0x7fdc28005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:08.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.388+0000 7fdc357fa640 1 -- 172.21.15.49:0/182740929 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdc3009c920 con 0x7fdc381014b0 2024-03-26T18:02:08.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.598+0000 7fdc3e86c640 1 -- 172.21.15.49:0/182740929 --> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fdc38061d00 con 0x7fdc200a2dd0 2024-03-26T18:02:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.604+0000 7fdc357fa640 1 -- 172.21.15.49:0/182740929 <== mgr.15516 v2:172.21.15.49:6800/2879593470 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fdc38061d00 con 0x7fdc200a2dd0 2024-03-26T18:02:08.585 INFO:teuthology.orchestra.run.smithi049.stdout:/ 1m 6m3h 2024-03-26T18:02:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 -- 172.21.15.49:0/182740929 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fdc200a2dd0 msgr2=0x7fdc200a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fdc200a2dd0 0x7fdc200a5290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc28011500 tx=0x7fdc28005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 -- 172.21.15.49:0/182740929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc381014b0 msgr2=0x7fdc381a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc381014b0 0x7fdc381a1370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009fd0 tx=0x7fdc3000ef10 comp rx=0 tx=0).stop 2024-03-26T18:02:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 -- 172.21.15.49:0/182740929 shutdown_connections 2024-03-26T18:02:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fdc200a2dd0 0x7fdc200a5290 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc381014b0 0x7fdc381a1370 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc38100b10 0x7fdc3819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 --2- 172.21.15.49:0/182740929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc381001b0 0x7fdc38199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 -- 172.21.15.49:0/182740929 >> 172.21.15.49:0/182740929 conn(0x7fdc380fbd50 msgr2=0x7fdc380fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 -- 172.21.15.49:0/182740929 shutdown_connections 2024-03-26T18:02:08.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:02:08.606+0000 7fdc3e86c640 1 -- 172.21.15.49:0/182740929 wait complete. 2024-03-26T18:02:08.598 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-26T18:02:08.631 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-26T18:02:08.642 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T18:02:08.665 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T18:02:08.665 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T18:02:08.665 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-03-26T18:02:08.719 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:08.741 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem Size Used Avail Use% Mounted on 2024-03-26T18:02:08.741 INFO:teuthology.orchestra.run.smithi049.stdout:0@5a3632a4-eb99-11ee-95d0-87774f69a715.cephfs=/volumes/qa/sv_1/af486460-0fa8-40db-9e10-2412dd703b0d 1.1T 356M 1.1T 1% /tmp/cephfs 2024-03-26T18:02:08.743 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-03-26T18:02:08.815 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-03-26T18:02:08.869 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-03-26T18:02:09.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: [26/Mar/2024:18:02:07] ENGINE Serving on http://172.21.15.49:8765 2024-03-26T18:02:09.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:02:09.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: [26/Mar/2024:18:02:07] ENGINE Serving on https://172.21.15.49:7150 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: [26/Mar/2024:18:02:07] ENGINE Bus STARTED 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:08 smithi115 ceph-mon[36174]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:02:09.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: [26/Mar/2024:18:02:07] ENGINE Serving on http://172.21.15.49:8765 2024-03-26T18:02:09.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:02:09.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: [26/Mar/2024:18:02:07] ENGINE Serving on https://172.21.15.49:7150 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: [26/Mar/2024:18:02:07] ENGINE Bus STARTED 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:08 smithi049 ceph-mon[32331]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:02:09.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: [26/Mar/2024:18:02:07] ENGINE Serving on http://172.21.15.49:8765 2024-03-26T18:02:09.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: from='client.25381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:02:09.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: [26/Mar/2024:18:02:07] ENGINE Serving on https://172.21.15.49:7150 2024-03-26T18:02:09.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: [26/Mar/2024:18:02:07] ENGINE Bus STARTED 2024-03-26T18:02:09.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:02:09.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-26T18:02:09.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:02:09.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:09.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:08 smithi171 ceph-mon[36033]: from='client.25399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi171 to 174.1M 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: Unable to set osd_memory_target on smithi171 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:02:10.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:09 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:02:10.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-26T18:02:10.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi171 to 174.1M 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: Unable to set osd_memory_target on smithi171 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:02:10.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:09 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:02:10.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-26T18:02:10.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi171 to 174.1M 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: Unable to set osd_memory_target on smithi171 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-26T18:02:10.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-26T18:02:10.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:10.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:02:10.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:02:10.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:09 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:02:10.669 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-03-26T18:02:10.720 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-03-26T18:02:10.773 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-26T18:02:10.821 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/: 2024-03-26T18:02:10.821 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:10.821 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:02:10.821 INFO:teuthology.orchestra.run.smithi049.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 .. 2024-03-26T18:02:10.821 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 data 2024-03-26T18:02:10.821 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:10.821 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data: 2024-03-26T18:02:10.822 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:10.822 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:10.822 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 .. 2024-03-26T18:02:10.823 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-03-26T18:02:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:[Service] 2024-03-26T18:02:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-03-26T18:02:10.884 INFO:teuthology.orchestra.run.smithi049.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-03-26T18:02:10.886 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-03-26T18:02:10.981 INFO:teuthology.orchestra.run.smithi049.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-03-26T18:02:11.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Unable to set osd_memory_target on smithi115 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:02:11.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:02:11.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:10 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-03-26T18:02:11.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Unable to set osd_memory_target on smithi115 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:02:11.252 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:02:11.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:02:11.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:02:11.253 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:10 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-03-26T18:02:11.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Unable to set osd_memory_target on smithi115 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-26T18:02:11.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi049:/etc/ceph/ceph.conf 2024-03-26T18:02:11.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi115:/etc/ceph/ceph.conf 2024-03-26T18:02:11.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi171:/etc/ceph/ceph.conf 2024-03-26T18:02:11.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: Updating smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.356 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:11.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:02:11.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:02:11.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:02:11.357 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:10 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:02:12.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:11 smithi115 ceph-mon[36174]: mgrmap e24: x(active, since 5s), standbys: z, y 2024-03-26T18:02:12.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:11 smithi049 ceph-mon[32331]: mgrmap e24: x(active, since 5s), standbys: z, y 2024-03-26T18:02:12.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:11 smithi171 ceph-mon[36033]: mgrmap e24: x(active, since 5s), standbys: z, y 2024-03-26T18:02:13.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:12 smithi115 ceph-mon[36174]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-26T18:02:13.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:12 smithi049 ceph-mon[32331]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-26T18:02:13.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:12 smithi171 ceph-mon[36033]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-26T18:02:15.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:14 smithi115 ceph-mon[36174]: pgmap v7: 129 pgs: 129 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 247 KiB/s wr, 86 op/s 2024-03-26T18:02:15.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:14 smithi115 ceph-mon[36174]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:15.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:14 smithi049 ceph-mon[32331]: pgmap v7: 129 pgs: 129 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 247 KiB/s wr, 86 op/s 2024-03-26T18:02:15.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:14 smithi049 ceph-mon[32331]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:15.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:14 smithi171 ceph-mon[36033]: pgmap v7: 129 pgs: 129 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 247 KiB/s wr, 86 op/s 2024-03-26T18:02:15.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:14 smithi171 ceph-mon[36033]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:16.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:15 smithi115 ceph-mon[36174]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:16.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:15 smithi049 ceph-mon[32331]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:16.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:15 smithi171 ceph-mon[36033]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:17.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:16 smithi115 ceph-mon[36174]: pgmap v8: 129 pgs: 129 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 192 KiB/s wr, 67 op/s 2024-03-26T18:02:17.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:16 smithi049 ceph-mon[32331]: pgmap v8: 129 pgs: 129 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 192 KiB/s wr, 67 op/s 2024-03-26T18:02:17.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:16 smithi171 ceph-mon[36033]: pgmap v8: 129 pgs: 129 active+clean; 2.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 192 KiB/s wr, 67 op/s 2024-03-26T18:02:18.966 INFO:teuthology.orchestra.run.smithi049.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-03-26T18:02:18.969 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-26T18:02:19.024 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/: 2024-03-26T18:02:19.024 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 .. 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 data 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data: 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:total 35 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 . 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 .. 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 base 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Mar 26 18:02 global 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 log 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_commit_ts 2024-03-26T18:02:19.025 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_dynshmem 2024-03-26T18:02:19.026 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Mar 26 18:02 pg_hba.conf 2024-03-26T18:02:19.026 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Mar 26 18:02 pg_ident.conf 2024-03-26T18:02:19.026 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 pg_logical 2024-03-26T18:02:19.026 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 pg_multixact 2024-03-26T18:02:19.026 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_notify 2024-03-26T18:02:19.026 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_replslot 2024-03-26T18:02:19.026 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_serial 2024-03-26T18:02:19.026 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_snapshots 2024-03-26T18:02:19.026 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_stat 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_stat_tmp 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 pg_subtrans 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_tblspc 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_twophase 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 PG_VERSION 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 pg_wal 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 pg_xact 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Mar 26 18:02 postgresql.auto.conf 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Mar 26 18:02 postgresql.conf 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base: 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.027 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 . 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:02 1 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:02 13493 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:02 13494 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/1: 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:total 7777 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:02 . 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 .. 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 112 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 113 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 1247 2024-03-26T18:02:19.028 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1247_fsm 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1247_vm 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:02 1249 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1249_fsm 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1249_vm 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:02 1255 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1255_fsm 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1255_vm 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:02 1259 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1259_fsm 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1259_vm 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:02 13344 2024-03-26T18:02:19.029 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13344_fsm 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13344_vm 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13346 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13348 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13349_fsm 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349_vm 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13351 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13353 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13354_fsm 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354_vm 2024-03-26T18:02:19.030 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13356 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13358 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13359_fsm 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359_vm 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13361 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13363 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1417 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1418 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 174 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 175 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2187 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2224 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2328 2024-03-26T18:02:19.031 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2336 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2337 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2579 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2600 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2600_fsm 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2600_vm 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2601_fsm 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601_vm 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2602 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2602_fsm 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2602_vm 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2603 2024-03-26T18:02:19.032 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2603_fsm 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2603_vm 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2604 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2605 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2605_fsm 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2605_vm 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2606_fsm 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606_vm 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2607 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2607_fsm 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2607_vm 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:02 2608 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2608_fsm 2024-03-26T18:02:19.033 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2608_vm 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:02 2609 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2609_fsm 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2609_vm 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2610 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2610_fsm 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2610_vm 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2611 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2612_fsm 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612_vm 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2613 2024-03-26T18:02:19.034 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615 2024-03-26T18:02:19.035 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2615_fsm 2024-03-26T18:02:19.035 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615_vm 2024-03-26T18:02:19.035 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2616 2024-03-26T18:02:19.035 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2616_fsm 2024-03-26T18:02:19.035 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2616_vm 2024-03-26T18:02:19.035 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:02 2617 2024-03-26T18:02:19.035 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2617_fsm 2024-03-26T18:02:19.035 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2617_vm 2024-03-26T18:02:19.035 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:02 2618 2024-03-26T18:02:19.035 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2618_fsm 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2618_vm 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 18:02 2619 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2619_fsm 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2619_vm 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2620 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2650 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2651 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2652 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2653 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2654 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2655 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2656 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2657 2024-03-26T18:02:19.036 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:02 2658 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 2659 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2660 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2661 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2662 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2663 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2664 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2665 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2666 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2667 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2668 2024-03-26T18:02:19.037 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2669 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2670 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:02 2673 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:02 2674 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:02 2675 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2678 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2679 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2680 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2681 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2682 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2683 2024-03-26T18:02:19.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2684 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2685 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2686 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2687 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2688 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2689 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:02 2690 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:02 2691 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2692 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2693 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2696 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2699 2024-03-26T18:02:19.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2701 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2702 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2703 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2704 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2753 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2753_fsm 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2753_vm 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2754 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2755 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2756 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2757 2024-03-26T18:02:19.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2830 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2831 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2832 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2833 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2834 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2835 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2836 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2837 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:02 2838 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2838_fsm 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2838_vm 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2839 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840_fsm 2024-03-26T18:02:19.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2840_vm 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2841 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2995 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2996 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3079_fsm 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079_vm 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3080 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3081 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3085 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3118 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3119 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:02 3164 2024-03-26T18:02:19.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3256 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3257 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3258 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3350 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3351 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3379 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3380 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3381 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3394 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3394_fsm 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3394_vm 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3395 2024-03-26T18:02:19.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3429 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3430 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3431 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3433 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3439 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3440 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3455 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:02 3456 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3456_fsm 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3456_vm 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3466 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3467 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3468 2024-03-26T18:02:19.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3501 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3502 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3503 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3534 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3541_fsm 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541_vm 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3542 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3574 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3575 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3576 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3596 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3597 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3598 2024-03-26T18:02:19.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3599 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3600_fsm 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600_vm 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3601_fsm 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601_vm 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3602_fsm 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602_vm 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603_fsm 2024-03-26T18:02:19.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3603_vm 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3604 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3605 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3606 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3607 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3608 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 3609 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3712 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3764_fsm 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764_vm 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3766 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3767 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3997 2024-03-26T18:02:19.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4143 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4144 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4145 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4146 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4147 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4148 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4149 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4150 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4151 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4152 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4153 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4154 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4155 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4156 2024-03-26T18:02:19.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4157 2024-03-26T18:02:19.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4158 2024-03-26T18:02:19.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4159 2024-03-26T18:02:19.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4160 2024-03-26T18:02:19.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4161 2024-03-26T18:02:19.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4162 2024-03-26T18:02:19.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4163 2024-03-26T18:02:19.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4164 2024-03-26T18:02:19.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4165 2024-03-26T18:02:19.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4166 2024-03-26T18:02:19.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4167 2024-03-26T18:02:19.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4168 2024-03-26T18:02:19.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4169 2024-03-26T18:02:19.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4170 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4171 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4172 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4173 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4174 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 5002 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 548 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 549 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6102 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6104 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6106 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6110 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6111 2024-03-26T18:02:19.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6112 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6113 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6117 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 826 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 827 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 828 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:02 pg_filenode.map 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 PG_VERSION 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:total 7777 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:02 . 2024-03-26T18:02:19.054 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 .. 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 112 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 113 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 1247 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1247_fsm 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1247_vm 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:02 1249 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1249_fsm 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1249_vm 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:02 1255 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1255_fsm 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1255_vm 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:02 1259 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1259_fsm 2024-03-26T18:02:19.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1259_vm 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:02 13344 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13344_fsm 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13344_vm 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13346 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13348 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13349_fsm 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349_vm 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13351 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13353 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13354_fsm 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354_vm 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13356 2024-03-26T18:02:19.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13358 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13359_fsm 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359_vm 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13361 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13363 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1417 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1418 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 174 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 175 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2187 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2224 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2328 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2336 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2337 2024-03-26T18:02:19.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2579 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2600 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2600_fsm 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2600_vm 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2601_fsm 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601_vm 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2602 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2602_fsm 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2602_vm 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2603 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2603_fsm 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2603_vm 2024-03-26T18:02:19.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2604 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2605 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2605_fsm 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2605_vm 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2606_fsm 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606_vm 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2607 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2607_fsm 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2607_vm 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:02 2608 2024-03-26T18:02:19.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2608_fsm 2024-03-26T18:02:19.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2608_vm 2024-03-26T18:02:19.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:02 2609 2024-03-26T18:02:19.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2609_fsm 2024-03-26T18:02:19.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2609_vm 2024-03-26T18:02:19.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2610 2024-03-26T18:02:19.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2610_fsm 2024-03-26T18:02:19.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2610_vm 2024-03-26T18:02:19.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2611 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2612_fsm 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612_vm 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2613 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2615_fsm 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615_vm 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2616 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2616_fsm 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2616_vm 2024-03-26T18:02:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:02 2617 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2617_fsm 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2617_vm 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:02 2618 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2618_fsm 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2618_vm 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 18:02 2619 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2619_fsm 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2619_vm 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2620 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2650 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2651 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2652 2024-03-26T18:02:19.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2653 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2654 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2655 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2656 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2657 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:02 2658 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 2659 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2660 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2661 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2662 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2663 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2664 2024-03-26T18:02:19.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2665 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2666 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2667 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2668 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2669 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2670 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:02 2673 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:02 2674 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:02 2675 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2678 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2679 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2680 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2681 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2682 2024-03-26T18:02:19.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2683 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2684 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2685 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2686 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2687 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2688 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2689 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:02 2690 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:02 2691 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2692 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2693 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2696 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2699 2024-03-26T18:02:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2701 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2702 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2703 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2704 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2753 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2753_fsm 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2753_vm 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2754 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2755 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2756 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2757 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2830 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2831 2024-03-26T18:02:19.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2832 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2833 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2834 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2835 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2836 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2837 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:02 2838 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2838_fsm 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2838_vm 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2839 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840_fsm 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2840_vm 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2841 2024-03-26T18:02:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2995 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2996 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3079_fsm 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079_vm 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3080 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3081 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3085 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3118 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3119 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:02 3164 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3256 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3257 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3258 2024-03-26T18:02:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3350 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3351 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3379 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3380 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3381 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3394 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3394_fsm 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3394_vm 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3395 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3429 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3430 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3431 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3433 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3439 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3440 2024-03-26T18:02:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3455 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:02 3456 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3456_fsm 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3456_vm 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3466 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3467 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3468 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3501 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3502 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3503 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3534 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3541_fsm 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541_vm 2024-03-26T18:02:19.070 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3542 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3574 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3575 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3576 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3596 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3597 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3598 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3599 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3600_fsm 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600_vm 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3601_fsm 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601_vm 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602 2024-03-26T18:02:19.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3602_fsm 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602_vm 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603_fsm 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3603_vm 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3604 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3605 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3606 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3607 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3608 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 3609 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3712 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3764_fsm 2024-03-26T18:02:19.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764_vm 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3766 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3767 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3997 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4143 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4144 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4145 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4146 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4147 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4148 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4149 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4150 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4151 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4152 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4153 2024-03-26T18:02:19.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4154 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4155 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4156 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4157 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4158 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4159 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4160 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4161 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4162 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4163 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4164 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4165 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4166 2024-03-26T18:02:19.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4167 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4168 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4169 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4170 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4171 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4172 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4173 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4174 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 5002 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 548 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 549 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6102 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6104 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6106 2024-03-26T18:02:19.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6110 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6111 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6112 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6113 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6117 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 826 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 827 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 828 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:02 pg_filenode.map 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 PG_VERSION 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:total 7777 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:02 . 2024-03-26T18:02:19.076 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 .. 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 112 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 113 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 1247 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1247_fsm 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1247_vm 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:02 1249 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1249_fsm 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1249_vm 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:02 1255 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1255_fsm 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1255_vm 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:02 1259 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1259_fsm 2024-03-26T18:02:19.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1259_vm 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:02 13344 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13344_fsm 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13344_vm 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13346 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13348 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13349_fsm 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349_vm 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13351 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13353 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13354_fsm 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354_vm 2024-03-26T18:02:19.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13356 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13358 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13359_fsm 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359_vm 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13361 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13363 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1417 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1418 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 174 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 175 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2187 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2224 2024-03-26T18:02:19.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2328 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2336 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2337 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2579 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2600 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2600_fsm 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2600_vm 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2601_fsm 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601_vm 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2602 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2602_fsm 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2602_vm 2024-03-26T18:02:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2603 2024-03-26T18:02:19.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2603_fsm 2024-03-26T18:02:19.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2603_vm 2024-03-26T18:02:19.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2604 2024-03-26T18:02:19.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2605 2024-03-26T18:02:19.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2605_fsm 2024-03-26T18:02:19.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2605_vm 2024-03-26T18:02:19.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606 2024-03-26T18:02:19.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2606_fsm 2024-03-26T18:02:19.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606_vm 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2607 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2607_fsm 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2607_vm 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:02 2608 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2608_fsm 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2608_vm 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:02 2609 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2609_fsm 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2609_vm 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2610 2024-03-26T18:02:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2610_fsm 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2610_vm 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2611 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2612_fsm 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612_vm 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2613 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2615_fsm 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615_vm 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2616 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2616_fsm 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2616_vm 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:02 2617 2024-03-26T18:02:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2617_fsm 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2617_vm 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:02 2618 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2618_fsm 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2618_vm 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 18:02 2619 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2619_fsm 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2619_vm 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2620 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2650 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2651 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2652 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2653 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2654 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2655 2024-03-26T18:02:19.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2656 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2657 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:02 2658 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 2659 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2660 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2661 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2662 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2663 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2664 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2665 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2666 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2667 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2668 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2669 2024-03-26T18:02:19.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2670 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:02 2673 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:02 2674 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:02 2675 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2678 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2679 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2680 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2681 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2682 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2683 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2684 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2685 2024-03-26T18:02:19.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2686 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2687 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2688 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2689 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:02 2690 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:02 2691 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2692 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2693 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2696 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2699 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2701 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2702 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2703 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2704 2024-03-26T18:02:19.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2753 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2753_fsm 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2753_vm 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2754 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2755 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2756 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2757 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2830 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2831 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2832 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2833 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2834 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2835 2024-03-26T18:02:19.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2836 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2837 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:02 2838 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2838_fsm 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2838_vm 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2839 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840_fsm 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2840_vm 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2841 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2995 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2996 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3079_fsm 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079_vm 2024-03-26T18:02:19.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3080 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3081 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3085 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3118 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3119 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:02 3164 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3256 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3257 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3258 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3350 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3351 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3379 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3380 2024-03-26T18:02:19.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3381 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3394 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3394_fsm 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3394_vm 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3395 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3429 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3430 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3431 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3433 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3439 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3440 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3455 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:02 3456 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3456_fsm 2024-03-26T18:02:19.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3456_vm 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3466 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3467 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3468 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3501 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3502 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3503 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3534 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3541_fsm 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541_vm 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3542 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3574 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3575 2024-03-26T18:02:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3576 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3596 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3597 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3598 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3599 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3600_fsm 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600_vm 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3601_fsm 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601_vm 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3602_fsm 2024-03-26T18:02:19.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602_vm 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603_fsm 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3603_vm 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3604 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3605 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3606 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3607 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3608 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 3609 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3712 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3764_fsm 2024-03-26T18:02:19.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764_vm 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3766 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3767 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3997 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4143 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4144 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4145 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4146 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4147 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4148 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4149 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4150 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4151 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4152 2024-03-26T18:02:19.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4153 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4154 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4155 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4156 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4157 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4158 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4159 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4160 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4161 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4162 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4163 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4164 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4165 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4166 2024-03-26T18:02:19.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4167 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4168 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4169 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4170 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4171 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4172 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4173 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4174 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 5002 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 548 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 549 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6102 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6104 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6106 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6110 2024-03-26T18:02:19.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6111 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6112 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6113 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6117 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 826 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 827 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 828 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:02 pg_filenode.map 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 PG_VERSION 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/global: 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:total 537 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Mar 26 18:02 . 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1213 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1213_fsm 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1213_vm 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1214 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1214_fsm 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1214_vm 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 1232 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 1233 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1260 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1260_fsm 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1260_vm 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1261 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1261_fsm 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1261_vm 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1262 2024-03-26T18:02:19.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1262_fsm 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1262_vm 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2396 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2396_fsm 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2396_vm 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2397 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2671 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2672 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2676 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2677 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2694 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2695 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2697 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2698 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2846 2024-03-26T18:02:19.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2847 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2964 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2965 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2966 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2967 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3592 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3593 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4060 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4061 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4175 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4176 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4177 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4178 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4181 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4182 2024-03-26T18:02:19.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4183 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4184 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4185 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4186 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6000 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6001 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6002 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6100 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6114 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6115 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 pg_control 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:02 pg_filenode.map 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/log: 2024-03-26T18:02:19.102 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.103 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.103 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.103 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-03-26T18:02:19.103 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.103 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.103 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.103 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.103 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:total 1 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 . 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 mappings 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Mar 26 18:02 replorigin_checkpoint 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 snapshots 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.104 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 .. 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 .. 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 . 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 members 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 offsets 2024-03-26T18:02:19.105 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 .. 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 0000 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 .. 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 0000 2024-03-26T18:02:19.106 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.107 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-03-26T18:02:19.108 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 0000 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.109 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:total 16384 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 . 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:02 000000010000000000000001 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 archive_status 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.110 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-03-26T18:02:19.111 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:02:19.111 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:02:19.111 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 .. 2024-03-26T18:02:19.111 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:02:19.111 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-03-26T18:02:19.111 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-03-26T18:02:19.111 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:02:19.111 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 18:02 .. 2024-03-26T18:02:19.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 0000 2024-03-26T18:02:19.111 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-03-26T18:02:19.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:18 smithi115 ceph-mon[36174]: pgmap v9: 129 pgs: 129 active+clean; 40 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 3.7 MiB/s wr, 132 op/s 2024-03-26T18:02:19.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:18 smithi049 ceph-mon[32331]: pgmap v9: 129 pgs: 129 active+clean; 40 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 3.7 MiB/s wr, 132 op/s 2024-03-26T18:02:19.269 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-03-26T18:02:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:dropping old tables... 2024-03-26T18:02:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-03-26T18:02:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-03-26T18:02:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-03-26T18:02:19.327 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-03-26T18:02:19.328 INFO:teuthology.orchestra.run.smithi049.stderr:creating tables... 2024-03-26T18:02:19.342 INFO:teuthology.orchestra.run.smithi049.stderr:generating data (client-side)... 2024-03-26T18:02:19.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:18 smithi171 ceph-mon[36033]: pgmap v9: 129 pgs: 129 active+clean; 40 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 3.7 MiB/s wr, 132 op/s 2024-03-26T18:02:19.423 INFO:teuthology.orchestra.run.smithi049.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.90 s) 2024-03-26T18:02:19.582 INFO:teuthology.orchestra.run.smithi049.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.22 s, remaining 3.30 s) 2024-03-26T18:02:19.668 INFO:teuthology.orchestra.run.smithi049.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.31 s, remaining 2.96 s) 2024-03-26T18:02:19.806 INFO:teuthology.orchestra.run.smithi049.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.44 s, remaining 3.11 s) 2024-03-26T18:02:19.952 INFO:teuthology.orchestra.run.smithi049.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.59 s, remaining 3.18 s) 2024-03-26T18:02:20.035 INFO:teuthology.orchestra.run.smithi049.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.67 s, remaining 2.91 s) 2024-03-26T18:02:20.175 INFO:teuthology.orchestra.run.smithi049.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.81 s, remaining 2.90 s) 2024-03-26T18:02:20.325 INFO:teuthology.orchestra.run.smithi049.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.96 s, remaining 2.89 s) 2024-03-26T18:02:20.420 INFO:teuthology.orchestra.run.smithi049.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.06 s, remaining 2.70 s) 2024-03-26T18:02:20.544 INFO:teuthology.orchestra.run.smithi049.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.18 s, remaining 2.60 s) 2024-03-26T18:02:20.712 INFO:teuthology.orchestra.run.smithi049.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.35 s, remaining 2.58 s) 2024-03-26T18:02:20.802 INFO:teuthology.orchestra.run.smithi049.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.44 s, remaining 2.40 s) 2024-03-26T18:02:20.940 INFO:teuthology.orchestra.run.smithi049.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.58 s, remaining 2.31 s) 2024-03-26T18:02:21.123 INFO:teuthology.orchestra.run.smithi049.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.76 s, remaining 2.26 s) 2024-03-26T18:02:21.182 INFO:teuthology.orchestra.run.smithi049.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.82 s, remaining 2.06 s) 2024-03-26T18:02:21.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:20 smithi115 ceph-mon[36174]: pgmap v10: 129 pgs: 129 active+clean; 40 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 3.3 MiB/s wr, 121 op/s 2024-03-26T18:02:21.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:20 smithi115 ceph-mon[36174]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:21.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:20 smithi049 ceph-mon[32331]: pgmap v10: 129 pgs: 129 active+clean; 40 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 3.3 MiB/s wr, 121 op/s 2024-03-26T18:02:21.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:20 smithi049 ceph-mon[32331]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:21.343 INFO:teuthology.orchestra.run.smithi049.stderr:1600000 of 3200000 tuples (50%) done (elapsed 1.98 s, remaining 1.98 s) 2024-03-26T18:02:21.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:20 smithi171 ceph-mon[36033]: pgmap v10: 129 pgs: 129 active+clean; 40 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 3.3 MiB/s wr, 121 op/s 2024-03-26T18:02:21.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:20 smithi171 ceph-mon[36033]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:21.480 INFO:teuthology.orchestra.run.smithi049.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.12 s, remaining 1.87 s) 2024-03-26T18:02:21.564 INFO:teuthology.orchestra.run.smithi049.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.20 s, remaining 1.71 s) 2024-03-26T18:02:21.709 INFO:teuthology.orchestra.run.smithi049.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.35 s, remaining 1.61 s) 2024-03-26T18:02:21.800 INFO:teuthology.orchestra.run.smithi049.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.44 s, remaining 1.46 s) 2024-03-26T18:02:21.925 INFO:teuthology.orchestra.run.smithi049.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.56 s, remaining 1.34 s) 2024-03-26T18:02:22.069 INFO:teuthology.orchestra.run.smithi049.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.71 s, remaining 1.23 s) 2024-03-26T18:02:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.80 s, remaining 1.10 s) 2024-03-26T18:02:22.296 INFO:teuthology.orchestra.run.smithi049.stderr:2400000 of 3200000 tuples (75%) done (elapsed 2.93 s, remaining 0.98 s) 2024-03-26T18:02:22.446 INFO:teuthology.orchestra.run.smithi049.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.08 s, remaining 0.86 s) 2024-03-26T18:02:22.520 INFO:teuthology.orchestra.run.smithi049.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.16 s, remaining 0.73 s) 2024-03-26T18:02:22.696 INFO:teuthology.orchestra.run.smithi049.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.33 s, remaining 0.62 s) 2024-03-26T18:02:22.760 INFO:teuthology.orchestra.run.smithi049.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.40 s, remaining 0.49 s) 2024-03-26T18:02:22.921 INFO:teuthology.orchestra.run.smithi049.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.56 s, remaining 0.37 s) 2024-03-26T18:02:23.054 INFO:teuthology.orchestra.run.smithi049.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.69 s, remaining 0.25 s) 2024-03-26T18:02:23.140 INFO:teuthology.orchestra.run.smithi049.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.78 s, remaining 0.12 s) 2024-03-26T18:02:23.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:22 smithi115 ceph-mon[36174]: pgmap v11: 129 pgs: 129 active+clean; 57 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 6.1 MiB/s wr, 124 op/s 2024-03-26T18:02:23.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:22 smithi049 ceph-mon[32331]: pgmap v11: 129 pgs: 129 active+clean; 57 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 6.1 MiB/s wr, 124 op/s 2024-03-26T18:02:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:3200000 of 3200000 tuples (100%) done (elapsed 3.90 s, remaining 0.00 s) 2024-03-26T18:02:23.307 INFO:teuthology.orchestra.run.smithi049.stderr:vacuuming... 2024-03-26T18:02:23.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:22 smithi171 ceph-mon[36033]: pgmap v11: 129 pgs: 129 active+clean; 57 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 6.1 MiB/s wr, 124 op/s 2024-03-26T18:02:23.936 INFO:teuthology.orchestra.run.smithi049.stderr:creating primary keys... 2024-03-26T18:02:25.124 INFO:teuthology.orchestra.run.smithi049.stderr:done in 5.80 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 3.96 s, vacuum 0.63 s, primary keys 1.19 s). 2024-03-26T18:02:25.127 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-03-26T18:02:25.208 INFO:teuthology.orchestra.run.smithi049.stderr:starting vacuum...end. 2024-03-26T18:02:25.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:24 smithi115 ceph-mon[36174]: pgmap v12: 129 pgs: 129 active+clean; 253 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 38 MiB/s wr, 170 op/s 2024-03-26T18:02:25.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:24 smithi049 ceph-mon[32331]: pgmap v12: 129 pgs: 129 active+clean; 253 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 38 MiB/s wr, 170 op/s 2024-03-26T18:02:25.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:24 smithi171 ceph-mon[36033]: pgmap v12: 129 pgs: 129 active+clean; 253 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 38 MiB/s wr, 170 op/s 2024-03-26T18:02:27.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:27 smithi115 ceph-mon[36174]: pgmap v13: 129 pgs: 129 active+clean; 253 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 38 MiB/s wr, 120 op/s 2024-03-26T18:02:27.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:26 smithi049 ceph-mon[32331]: pgmap v13: 129 pgs: 129 active+clean; 253 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 38 MiB/s wr, 120 op/s 2024-03-26T18:02:27.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:27 smithi171 ceph-mon[36033]: pgmap v13: 129 pgs: 129 active+clean; 253 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 38 MiB/s wr, 120 op/s 2024-03-26T18:02:29.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:29 smithi115 ceph-mon[36174]: pgmap v14: 129 pgs: 129 active+clean; 514 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 76 MiB/s wr, 192 op/s 2024-03-26T18:02:29.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:29 smithi049 ceph-mon[32331]: pgmap v14: 129 pgs: 129 active+clean; 514 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 76 MiB/s wr, 192 op/s 2024-03-26T18:02:29.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:29 smithi171 ceph-mon[36033]: pgmap v14: 129 pgs: 129 active+clean; 514 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 76 MiB/s wr, 192 op/s 2024-03-26T18:02:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 5.0 s, 3705.8 tps, lat 26.449 ms stddev 29.967 2024-03-26T18:02:31.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:31 smithi049 ceph-mon[32331]: pgmap v15: 129 pgs: 129 active+clean; 514 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 73 MiB/s wr, 121 op/s 2024-03-26T18:02:31.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:31 smithi171 ceph-mon[36033]: pgmap v15: 129 pgs: 129 active+clean; 514 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 73 MiB/s wr, 121 op/s 2024-03-26T18:02:31.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:31 smithi115 ceph-mon[36174]: pgmap v15: 129 pgs: 129 active+clean; 514 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 73 MiB/s wr, 121 op/s 2024-03-26T18:02:33.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:33 smithi049 ceph-mon[32331]: pgmap v16: 129 pgs: 129 active+clean; 553 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 73 MiB/s wr, 136 op/s 2024-03-26T18:02:33.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:33 smithi171 ceph-mon[36033]: pgmap v16: 129 pgs: 129 active+clean; 553 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 73 MiB/s wr, 136 op/s 2024-03-26T18:02:33.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:33 smithi115 ceph-mon[36174]: pgmap v16: 129 pgs: 129 active+clean; 553 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 73 MiB/s wr, 136 op/s 2024-03-26T18:02:35.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 10.0 s, 3305.0 tps, lat 30.340 ms stddev 31.124 2024-03-26T18:02:35.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:35 smithi049 ceph-mon[32331]: pgmap v17: 129 pgs: 129 active+clean; 900 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 73 MiB/s wr, 396 op/s 2024-03-26T18:02:35.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:35 smithi171 ceph-mon[36033]: pgmap v17: 129 pgs: 129 active+clean; 900 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 73 MiB/s wr, 396 op/s 2024-03-26T18:02:35.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:35 smithi115 ceph-mon[36174]: pgmap v17: 129 pgs: 129 active+clean; 900 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 73 MiB/s wr, 396 op/s 2024-03-26T18:02:37.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:37 smithi049 ceph-mon[32331]: pgmap v18: 129 pgs: 129 active+clean; 900 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 MiB/s wr, 349 op/s 2024-03-26T18:02:37.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:37 smithi171 ceph-mon[36033]: pgmap v18: 129 pgs: 129 active+clean; 900 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 MiB/s wr, 349 op/s 2024-03-26T18:02:37.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:37 smithi115 ceph-mon[36174]: pgmap v18: 129 pgs: 129 active+clean; 900 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 MiB/s wr, 349 op/s 2024-03-26T18:02:39.251 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:39 smithi049 ceph-mon[32331]: pgmap v19: 129 pgs: 129 active+clean; 933 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 46 MiB/s wr, 659 op/s 2024-03-26T18:02:39.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:39 smithi115 ceph-mon[36174]: pgmap v19: 129 pgs: 129 active+clean; 933 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 46 MiB/s wr, 659 op/s 2024-03-26T18:02:39.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:39 smithi171 ceph-mon[36033]: pgmap v19: 129 pgs: 129 active+clean; 933 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 46 MiB/s wr, 659 op/s 2024-03-26T18:02:40.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 15.0 s, 3352.2 tps, lat 29.811 ms stddev 32.448 2024-03-26T18:02:41.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:41 smithi171 ceph-mon[36033]: pgmap v20: 129 pgs: 129 active+clean; 933 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.5 MiB/s wr, 587 op/s 2024-03-26T18:02:41.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:41 smithi115 ceph-mon[36174]: pgmap v20: 129 pgs: 129 active+clean; 933 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.5 MiB/s wr, 587 op/s 2024-03-26T18:02:41.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:41 smithi049 ceph-mon[32331]: pgmap v20: 129 pgs: 129 active+clean; 933 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.5 MiB/s wr, 587 op/s 2024-03-26T18:02:43.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:43 smithi171 ceph-mon[36033]: pgmap v21: 129 pgs: 129 active+clean; 934 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 MiB/s wr, 605 op/s 2024-03-26T18:02:43.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:43 smithi115 ceph-mon[36174]: pgmap v21: 129 pgs: 129 active+clean; 934 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 MiB/s wr, 605 op/s 2024-03-26T18:02:43.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:43 smithi049 ceph-mon[32331]: pgmap v21: 129 pgs: 129 active+clean; 934 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 MiB/s wr, 605 op/s 2024-03-26T18:02:45.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 20.0 s, 3378.4 tps, lat 29.555 ms stddev 30.648 2024-03-26T18:02:45.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:45 smithi171 ceph-mon[36033]: pgmap v22: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 879 op/s 2024-03-26T18:02:45.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:45 smithi115 ceph-mon[36174]: pgmap v22: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 879 op/s 2024-03-26T18:02:45.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:45 smithi049 ceph-mon[32331]: pgmap v22: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 879 op/s 2024-03-26T18:02:47.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:47 smithi171 ceph-mon[36033]: pgmap v23: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.6 MiB/s wr, 617 op/s 2024-03-26T18:02:47.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:47 smithi115 ceph-mon[36174]: pgmap v23: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.6 MiB/s wr, 617 op/s 2024-03-26T18:02:47.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:47 smithi049 ceph-mon[32331]: pgmap v23: 129 pgs: 129 active+clean; 938 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.6 MiB/s wr, 617 op/s 2024-03-26T18:02:49.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:49 smithi171 ceph-mon[36033]: pgmap v24: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 924 op/s 2024-03-26T18:02:49.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:49 smithi115 ceph-mon[36174]: pgmap v24: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 924 op/s 2024-03-26T18:02:49.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:49 smithi049 ceph-mon[32331]: pgmap v24: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 924 op/s 2024-03-26T18:02:50.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 25.0 s, 2941.6 tps, lat 34.019 ms stddev 54.984 2024-03-26T18:02:51.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:51 smithi171 ceph-mon[36033]: pgmap v25: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.5 MiB/s wr, 613 op/s 2024-03-26T18:02:51.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:51 smithi115 ceph-mon[36174]: pgmap v25: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.5 MiB/s wr, 613 op/s 2024-03-26T18:02:51.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:51 smithi049 ceph-mon[32331]: pgmap v25: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.5 MiB/s wr, 613 op/s 2024-03-26T18:02:53.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:53 smithi171 ceph-mon[36033]: pgmap v26: 129 pgs: 129 active+clean; 960 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 649 op/s 2024-03-26T18:02:53.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:53 smithi115 ceph-mon[36174]: pgmap v26: 129 pgs: 129 active+clean; 960 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 649 op/s 2024-03-26T18:02:53.502 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:53 smithi049 ceph-mon[32331]: pgmap v26: 129 pgs: 129 active+clean; 960 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 649 op/s 2024-03-26T18:02:55.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 30.0 s, 3051.6 tps, lat 32.672 ms stddev 35.716 2024-03-26T18:02:55.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:55 smithi171 ceph-mon[36033]: pgmap v27: 129 pgs: 129 active+clean; 964 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2024-03-26T18:02:55.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:55 smithi115 ceph-mon[36174]: pgmap v27: 129 pgs: 129 active+clean; 964 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2024-03-26T18:02:55.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:55 smithi049 ceph-mon[32331]: pgmap v27: 129 pgs: 129 active+clean; 964 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2024-03-26T18:02:57.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:57 smithi171 ceph-mon[36033]: pgmap v28: 129 pgs: 129 active+clean; 964 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 800 op/s 2024-03-26T18:02:57.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:57 smithi115 ceph-mon[36174]: pgmap v28: 129 pgs: 129 active+clean; 964 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 800 op/s 2024-03-26T18:02:57.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:57 smithi049 ceph-mon[32331]: pgmap v28: 129 pgs: 129 active+clean; 964 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 800 op/s 2024-03-26T18:02:59.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:02:59 smithi171 ceph-mon[36033]: pgmap v29: 129 pgs: 129 active+clean; 981 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 MiB/s wr, 1.11k op/s 2024-03-26T18:02:59.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:02:59 smithi115 ceph-mon[36174]: pgmap v29: 129 pgs: 129 active+clean; 981 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 MiB/s wr, 1.11k op/s 2024-03-26T18:02:59.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:02:59 smithi049 ceph-mon[32331]: pgmap v29: 129 pgs: 129 active+clean; 981 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 MiB/s wr, 1.11k op/s 2024-03-26T18:03:00.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 35.0 s, 3277.4 tps, lat 30.699 ms stddev 33.354 2024-03-26T18:03:01.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:01 smithi049 ceph-mon[32331]: pgmap v30: 129 pgs: 129 active+clean; 981 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 800 op/s 2024-03-26T18:03:01.251 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:03:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:03:00.863+0000 7f34775a0700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:03:01.251 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:03:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:03:00.863+0000 7f34775a0700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:03:01.251 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:03:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:03:00.863+0000 7f34775a0700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:03:01.251 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:03:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:03:00.863+0000 7f34775a0700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:03:01.251 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:03:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:03:00.863+0000 7f34775a0700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T18:03:01.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:01 smithi171 ceph-mon[36033]: pgmap v30: 129 pgs: 129 active+clean; 981 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 800 op/s 2024-03-26T18:03:01.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:01 smithi115 ceph-mon[36174]: pgmap v30: 129 pgs: 129 active+clean; 981 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 800 op/s 2024-03-26T18:03:03.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:03 smithi171 ceph-mon[36033]: pgmap v31: 129 pgs: 129 active+clean; 981 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 864 op/s 2024-03-26T18:03:03.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:03 smithi115 ceph-mon[36174]: pgmap v31: 129 pgs: 129 active+clean; 981 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 864 op/s 2024-03-26T18:03:03.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:03 smithi049 ceph-mon[32331]: pgmap v31: 129 pgs: 129 active+clean; 981 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 864 op/s 2024-03-26T18:03:04.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:04 smithi171 ceph-mon[36033]: mgrmap e25: x(active, since 57s), standbys: z, y 2024-03-26T18:03:04.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:04 smithi115 ceph-mon[36174]: mgrmap e25: x(active, since 57s), standbys: z, y 2024-03-26T18:03:04.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:04 smithi049 ceph-mon[32331]: mgrmap e25: x(active, since 57s), standbys: z, y 2024-03-26T18:03:05.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 40.0 s, 570.2 tps, lat 166.083 ms stddev 267.969 2024-03-26T18:03:05.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:05 smithi171 ceph-mon[36033]: pgmap v32: 129 pgs: 129 active+clean; 982 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-03-26T18:03:05.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:05 smithi115 ceph-mon[36174]: pgmap v32: 129 pgs: 129 active+clean; 982 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-03-26T18:03:05.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:05 smithi049 ceph-mon[32331]: pgmap v32: 129 pgs: 129 active+clean; 982 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-03-26T18:03:07.318 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:07 smithi049 ceph-mon[32331]: pgmap v33: 129 pgs: 129 active+clean; 982 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 592 op/s 2024-03-26T18:03:07.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:07 smithi171 ceph-mon[36033]: pgmap v33: 129 pgs: 129 active+clean; 982 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 592 op/s 2024-03-26T18:03:07.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:07 smithi115 ceph-mon[36174]: pgmap v33: 129 pgs: 129 active+clean; 982 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 592 op/s 2024-03-26T18:03:09.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:09 smithi049 ceph-mon[32331]: pgmap v34: 129 pgs: 129 active+clean; 985 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-03-26T18:03:09.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:09 smithi171 ceph-mon[36033]: pgmap v34: 129 pgs: 129 active+clean; 985 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-03-26T18:03:09.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:09 smithi115 ceph-mon[36174]: pgmap v34: 129 pgs: 129 active+clean; 985 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-03-26T18:03:10.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 45.0 s, 267.6 tps, lat 336.816 ms stddev 214.378 2024-03-26T18:03:11.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:11 smithi171 ceph-mon[36033]: pgmap v35: 129 pgs: 129 active+clean; 985 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 737 op/s 2024-03-26T18:03:11.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:11 smithi115 ceph-mon[36174]: pgmap v35: 129 pgs: 129 active+clean; 985 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 737 op/s 2024-03-26T18:03:11.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:11 smithi049 ceph-mon[32331]: pgmap v35: 129 pgs: 129 active+clean; 985 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 737 op/s 2024-03-26T18:03:12.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:12 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:03:12.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:12 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:12 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:12 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:12 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:12 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:03:12.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:12 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:12 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:12 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:12 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:12 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:03:12.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:12 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:12 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:12 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:12.501 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:12 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:13.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:13 smithi171 ceph-mon[36033]: pgmap v36: 129 pgs: 129 active+clean; 989 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.03k op/s 2024-03-26T18:03:13.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:13 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:03:13.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:13 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:03:13.355 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:13 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:13.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:13 smithi115 ceph-mon[36174]: pgmap v36: 129 pgs: 129 active+clean; 989 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.03k op/s 2024-03-26T18:03:13.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:13 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:03:13.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:13 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:03:13.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:13 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:13.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:13 smithi049 ceph-mon[32331]: pgmap v36: 129 pgs: 129 active+clean; 989 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.03k op/s 2024-03-26T18:03:13.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:13 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:03:13.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:13 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:03:13.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:13 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:03:15.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 50.0 s, 1927.0 tps, lat 59.145 ms stddev 131.166 2024-03-26T18:03:15.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:15 smithi171 ceph-mon[36033]: pgmap v37: 129 pgs: 129 active+clean; 994 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 MiB/s wr, 1.36k op/s 2024-03-26T18:03:15.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:15 smithi115 ceph-mon[36174]: pgmap v37: 129 pgs: 129 active+clean; 994 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 MiB/s wr, 1.36k op/s 2024-03-26T18:03:15.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:15 smithi049 ceph-mon[32331]: pgmap v37: 129 pgs: 129 active+clean; 994 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 MiB/s wr, 1.36k op/s 2024-03-26T18:03:17.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:17 smithi171 ceph-mon[36033]: pgmap v38: 129 pgs: 129 active+clean; 994 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 MiB/s wr, 1.13k op/s 2024-03-26T18:03:17.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:17 smithi115 ceph-mon[36174]: pgmap v38: 129 pgs: 129 active+clean; 994 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 MiB/s wr, 1.13k op/s 2024-03-26T18:03:17.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:17 smithi049 ceph-mon[32331]: pgmap v38: 129 pgs: 129 active+clean; 994 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 MiB/s wr, 1.13k op/s 2024-03-26T18:03:19.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:19 smithi171 ceph-mon[36033]: pgmap v39: 129 pgs: 129 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 1.41k op/s 2024-03-26T18:03:19.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:19 smithi115 ceph-mon[36174]: pgmap v39: 129 pgs: 129 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 1.41k op/s 2024-03-26T18:03:19.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:19 smithi049 ceph-mon[32331]: pgmap v39: 129 pgs: 129 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 35 MiB/s wr, 1.41k op/s 2024-03-26T18:03:20.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 55.0 s, 2479.2 tps, lat 40.596 ms stddev 48.594 2024-03-26T18:03:21.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:21 smithi171 ceph-mon[36033]: pgmap v40: 129 pgs: 129 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 960 op/s 2024-03-26T18:03:21.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:21 smithi115 ceph-mon[36174]: pgmap v40: 129 pgs: 129 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 960 op/s 2024-03-26T18:03:21.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:21 smithi049 ceph-mon[32331]: pgmap v40: 129 pgs: 129 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 960 op/s 2024-03-26T18:03:23.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:23 smithi171 ceph-mon[36033]: pgmap v41: 129 pgs: 129 active+clean; 1.0 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 1.10k op/s 2024-03-26T18:03:23.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:23 smithi115 ceph-mon[36174]: pgmap v41: 129 pgs: 129 active+clean; 1.0 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 1.10k op/s 2024-03-26T18:03:23.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:23 smithi049 ceph-mon[32331]: pgmap v41: 129 pgs: 129 active+clean; 1.0 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 1.10k op/s 2024-03-26T18:03:25.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 60.0 s, 2906.8 tps, lat 34.445 ms stddev 37.157 2024-03-26T18:03:25.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:25 smithi171 ceph-mon[36033]: pgmap v42: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-03-26T18:03:25.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:25 smithi115 ceph-mon[36174]: pgmap v42: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-03-26T18:03:25.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:25 smithi049 ceph-mon[32331]: pgmap v42: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-03-26T18:03:27.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:27 smithi171 ceph-mon[36033]: pgmap v43: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.3 MiB/s wr, 704 op/s 2024-03-26T18:03:27.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:27 smithi115 ceph-mon[36174]: pgmap v43: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.3 MiB/s wr, 704 op/s 2024-03-26T18:03:27.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:27 smithi049 ceph-mon[32331]: pgmap v43: 129 pgs: 129 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.3 MiB/s wr, 704 op/s 2024-03-26T18:03:29.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:29 smithi171 ceph-mon[36033]: pgmap v44: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 986 op/s 2024-03-26T18:03:29.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:29 smithi115 ceph-mon[36174]: pgmap v44: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 986 op/s 2024-03-26T18:03:29.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:29 smithi049 ceph-mon[32331]: pgmap v44: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 986 op/s 2024-03-26T18:03:30.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 65.0 s, 2450.0 tps, lat 40.557 ms stddev 46.013 2024-03-26T18:03:31.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:31 smithi171 ceph-mon[36033]: pgmap v45: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.0 MiB/s wr, 708 op/s 2024-03-26T18:03:31.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:31 smithi115 ceph-mon[36174]: pgmap v45: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.0 MiB/s wr, 708 op/s 2024-03-26T18:03:31.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:31 smithi049 ceph-mon[32331]: pgmap v45: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.0 MiB/s wr, 708 op/s 2024-03-26T18:03:33.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:33 smithi115 ceph-mon[36174]: pgmap v46: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.3 MiB/s wr, 785 op/s 2024-03-26T18:03:33.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:33 smithi049 ceph-mon[32331]: pgmap v46: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.3 MiB/s wr, 785 op/s 2024-03-26T18:03:33.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:33 smithi171 ceph-mon[36033]: pgmap v46: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.3 MiB/s wr, 785 op/s 2024-03-26T18:03:35.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 70.0 s, 2587.2 tps, lat 38.915 ms stddev 41.747 2024-03-26T18:03:35.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:35 smithi115 ceph-mon[36174]: pgmap v47: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.6 MiB/s wr, 847 op/s 2024-03-26T18:03:35.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:35 smithi049 ceph-mon[32331]: pgmap v47: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.6 MiB/s wr, 847 op/s 2024-03-26T18:03:35.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:35 smithi171 ceph-mon[36033]: pgmap v47: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.6 MiB/s wr, 847 op/s 2024-03-26T18:03:37.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:37 smithi115 ceph-mon[36174]: pgmap v48: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 558 op/s 2024-03-26T18:03:37.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:37 smithi049 ceph-mon[32331]: pgmap v48: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 558 op/s 2024-03-26T18:03:37.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:37 smithi171 ceph-mon[36033]: pgmap v48: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 558 op/s 2024-03-26T18:03:38.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:38 smithi115 ceph-mon[36174]: mgrmap e26: x(active, since 91s), standbys: z, y 2024-03-26T18:03:38.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:38 smithi049 ceph-mon[32331]: mgrmap e26: x(active, since 91s), standbys: z, y 2024-03-26T18:03:38.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:38 smithi171 ceph-mon[36033]: mgrmap e26: x(active, since 91s), standbys: z, y 2024-03-26T18:03:39.385 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:39 smithi115 ceph-mon[36174]: pgmap v49: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.3 MiB/s wr, 844 op/s 2024-03-26T18:03:39.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:39 smithi049 ceph-mon[32331]: pgmap v49: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.3 MiB/s wr, 844 op/s 2024-03-26T18:03:39.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:39 smithi171 ceph-mon[36033]: pgmap v49: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.3 MiB/s wr, 844 op/s 2024-03-26T18:03:40.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 75.0 s, 2314.6 tps, lat 43.084 ms stddev 50.195 2024-03-26T18:03:41.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:41 smithi115 ceph-mon[36174]: pgmap v50: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.6 MiB/s wr, 562 op/s 2024-03-26T18:03:41.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:41 smithi049 ceph-mon[32331]: pgmap v50: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.6 MiB/s wr, 562 op/s 2024-03-26T18:03:41.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:41 smithi171 ceph-mon[36033]: pgmap v50: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.6 MiB/s wr, 562 op/s 2024-03-26T18:03:43.735 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:43 smithi115 ceph-mon[36174]: pgmap v51: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 MiB/s wr, 613 op/s 2024-03-26T18:03:43.750 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:43 smithi049 ceph-mon[32331]: pgmap v51: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 MiB/s wr, 613 op/s 2024-03-26T18:03:43.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:43 smithi171 ceph-mon[36033]: pgmap v51: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 MiB/s wr, 613 op/s 2024-03-26T18:03:45.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 80.0 s, 2091.2 tps, lat 47.848 ms stddev 69.674 2024-03-26T18:03:45.735 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:45 smithi115 ceph-mon[36174]: pgmap v52: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-03-26T18:03:45.750 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:45 smithi049 ceph-mon[32331]: pgmap v52: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-03-26T18:03:45.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:45 smithi171 ceph-mon[36033]: pgmap v52: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-03-26T18:03:47.750 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:47 smithi049 ceph-mon[32331]: pgmap v53: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 832 op/s 2024-03-26T18:03:47.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:47 smithi171 ceph-mon[36033]: pgmap v53: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 832 op/s 2024-03-26T18:03:47.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:47 smithi115 ceph-mon[36174]: pgmap v53: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 832 op/s 2024-03-26T18:03:49.750 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:49 smithi049 ceph-mon[32331]: pgmap v54: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-03-26T18:03:49.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:49 smithi171 ceph-mon[36033]: pgmap v54: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-03-26T18:03:49.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:49 smithi115 ceph-mon[36174]: pgmap v54: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-03-26T18:03:50.208 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 85.0 s, 1611.2 tps, lat 61.883 ms stddev 76.408 2024-03-26T18:03:51.750 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:51 smithi049 ceph-mon[32331]: pgmap v55: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 814 op/s 2024-03-26T18:03:51.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:51 smithi171 ceph-mon[36033]: pgmap v55: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 814 op/s 2024-03-26T18:03:51.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:51 smithi115 ceph-mon[36174]: pgmap v55: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 814 op/s 2024-03-26T18:03:53.750 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:53 smithi049 ceph-mon[32331]: pgmap v56: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 875 op/s 2024-03-26T18:03:53.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:53 smithi171 ceph-mon[36033]: pgmap v56: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 875 op/s 2024-03-26T18:03:53.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:53 smithi115 ceph-mon[36174]: pgmap v56: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 875 op/s 2024-03-26T18:03:55.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 90.0 s, 1373.4 tps, lat 73.052 ms stddev 83.885 2024-03-26T18:03:55.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:55 smithi171 ceph-mon[36033]: pgmap v57: 129 pgs: 129 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2024-03-26T18:03:55.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:55 smithi115 ceph-mon[36174]: pgmap v57: 129 pgs: 129 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2024-03-26T18:03:56.000 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:55 smithi049 ceph-mon[32331]: pgmap v57: 129 pgs: 129 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2024-03-26T18:03:57.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:57 smithi171 ceph-mon[36033]: pgmap v58: 129 pgs: 129 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 MiB/s wr, 790 op/s 2024-03-26T18:03:57.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:57 smithi115 ceph-mon[36174]: pgmap v58: 129 pgs: 129 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 MiB/s wr, 790 op/s 2024-03-26T18:03:58.000 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:57 smithi049 ceph-mon[32331]: pgmap v58: 129 pgs: 129 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 MiB/s wr, 790 op/s 2024-03-26T18:03:59.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:03:59 smithi171 ceph-mon[36033]: pgmap v59: 129 pgs: 129 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-03-26T18:03:59.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:03:59 smithi115 ceph-mon[36174]: pgmap v59: 129 pgs: 129 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-03-26T18:04:00.000 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:03:59 smithi049 ceph-mon[32331]: pgmap v59: 129 pgs: 129 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-03-26T18:04:00.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 95.0 s, 1866.6 tps, lat 53.648 ms stddev 58.281 2024-03-26T18:04:01.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:01 smithi171 ceph-mon[36033]: pgmap v60: 129 pgs: 129 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.8 MiB/s wr, 777 op/s 2024-03-26T18:04:01.984 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:01 smithi115 ceph-mon[36174]: pgmap v60: 129 pgs: 129 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.8 MiB/s wr, 777 op/s 2024-03-26T18:04:01.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:01 smithi049 ceph-mon[32331]: pgmap v60: 129 pgs: 129 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.8 MiB/s wr, 777 op/s 2024-03-26T18:04:03.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:03 smithi171 ceph-mon[36033]: pgmap v61: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 852 op/s 2024-03-26T18:04:03.984 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:03 smithi115 ceph-mon[36174]: pgmap v61: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 852 op/s 2024-03-26T18:04:03.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:03 smithi049 ceph-mon[32331]: pgmap v61: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 852 op/s 2024-03-26T18:04:05.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 100.0 s, 611.4 tps, lat 157.455 ms stddev 213.831 2024-03-26T18:04:05.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:05 smithi171 ceph-mon[36033]: pgmap v62: 129 pgs: 129 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 973 op/s 2024-03-26T18:04:05.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:05 smithi115 ceph-mon[36174]: pgmap v62: 129 pgs: 129 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 973 op/s 2024-03-26T18:04:05.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:05 smithi049 ceph-mon[32331]: pgmap v62: 129 pgs: 129 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 973 op/s 2024-03-26T18:04:07.802 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:07 smithi049 ceph-mon[32331]: pgmap v63: 129 pgs: 129 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 512 op/s 2024-03-26T18:04:07.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:07 smithi171 ceph-mon[36033]: pgmap v63: 129 pgs: 129 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 512 op/s 2024-03-26T18:04:07.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:07 smithi115 ceph-mon[36174]: pgmap v63: 129 pgs: 129 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 512 op/s 2024-03-26T18:04:09.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:09 smithi115 ceph-mon[36174]: pgmap v64: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.5 MiB/s wr, 861 op/s 2024-03-26T18:04:09.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:09 smithi171 ceph-mon[36033]: pgmap v64: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.5 MiB/s wr, 861 op/s 2024-03-26T18:04:09.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:09 smithi049 ceph-mon[32331]: pgmap v64: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.5 MiB/s wr, 861 op/s 2024-03-26T18:04:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 105.0 s, 466.4 tps, lat 221.918 ms stddev 337.374 2024-03-26T18:04:11.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:11 smithi171 ceph-mon[36033]: pgmap v65: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-03-26T18:04:11.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:11 smithi115 ceph-mon[36174]: pgmap v65: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-03-26T18:04:11.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:11 smithi049 ceph-mon[32331]: pgmap v65: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-03-26T18:04:13.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:13 smithi171 ceph-mon[36033]: pgmap v66: 129 pgs: 129 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 887 op/s 2024-03-26T18:04:13.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:13 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:04:13.984 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:13 smithi115 ceph-mon[36174]: pgmap v66: 129 pgs: 129 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 887 op/s 2024-03-26T18:04:13.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:13 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:04:13.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:13 smithi049 ceph-mon[32331]: pgmap v66: 129 pgs: 129 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 887 op/s 2024-03-26T18:04:13.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:13 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 110.0 s, 2226.8 tps, lat 44.893 ms stddev 52.627 2024-03-26T18:04:15.234 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:14 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:04:15.234 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:14 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:04:15.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:14 smithi115 ceph-mon[36174]: pgmap v67: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2024-03-26T18:04:15.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:14 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:04:15.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:14 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:04:15.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:14 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:04:15.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:14 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:04:15.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:14 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:04:15.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:14 smithi049 ceph-mon[32331]: pgmap v67: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2024-03-26T18:04:15.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:14 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:04:15.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:14 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:04:15.250 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:14 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:04:15.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:14 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:04:15.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:14 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:04:15.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:14 smithi171 ceph-mon[36033]: pgmap v67: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2024-03-26T18:04:15.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:14 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:04:15.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:14 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:04:15.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:14 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:04:17.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:17 smithi171 ceph-mon[36033]: pgmap v68: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 1.30k op/s 2024-03-26T18:04:17.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:17 smithi115 ceph-mon[36174]: pgmap v68: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 1.30k op/s 2024-03-26T18:04:17.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:17 smithi049 ceph-mon[32331]: pgmap v68: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 1.30k op/s 2024-03-26T18:04:19.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:19 smithi171 ceph-mon[36033]: pgmap v69: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2024-03-26T18:04:19.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:19 smithi115 ceph-mon[36174]: pgmap v69: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2024-03-26T18:04:19.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:19 smithi049 ceph-mon[32331]: pgmap v69: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2024-03-26T18:04:20.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 115.0 s, 1997.0 tps, lat 49.745 ms stddev 54.630 2024-03-26T18:04:21.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:21 smithi171 ceph-mon[36033]: pgmap v70: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-03-26T18:04:21.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:21 smithi115 ceph-mon[36174]: pgmap v70: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-03-26T18:04:21.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:21 smithi049 ceph-mon[32331]: pgmap v70: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-03-26T18:04:23.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:23 smithi171 ceph-mon[36033]: pgmap v71: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 1.32k op/s 2024-03-26T18:04:23.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:23 smithi115 ceph-mon[36174]: pgmap v71: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 1.32k op/s 2024-03-26T18:04:23.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:23 smithi049 ceph-mon[32331]: pgmap v71: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 1.32k op/s 2024-03-26T18:04:25.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 120.0 s, 1926.4 tps, lat 52.356 ms stddev 57.428 2024-03-26T18:04:25.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:25 smithi171 ceph-mon[36033]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 1.27k op/s 2024-03-26T18:04:25.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:25 smithi115 ceph-mon[36174]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 1.27k op/s 2024-03-26T18:04:25.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:25 smithi049 ceph-mon[32331]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 1.27k op/s 2024-03-26T18:04:26.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:26 smithi171 ceph-mon[36033]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:04:26.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:26 smithi115 ceph-mon[36174]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:04:26.500 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:26 smithi049 ceph-mon[32331]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:04:27.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:27 smithi171 ceph-mon[36033]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-03-26T18:04:27.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:27 smithi115 ceph-mon[36174]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-03-26T18:04:27.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:27 smithi049 ceph-mon[32331]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-03-26T18:04:29.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:29 smithi171 ceph-mon[36033]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.9 MiB/s wr, 859 op/s 2024-03-26T18:04:29.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:29 smithi115 ceph-mon[36174]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.9 MiB/s wr, 859 op/s 2024-03-26T18:04:29.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:29 smithi049 ceph-mon[32331]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.9 MiB/s wr, 859 op/s 2024-03-26T18:04:30.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 125.0 s, 1708.0 tps, lat 58.247 ms stddev 66.694 2024-03-26T18:04:31.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:31 smithi171 ceph-mon[36033]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 587 op/s 2024-03-26T18:04:31.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:31 smithi115 ceph-mon[36174]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 587 op/s 2024-03-26T18:04:31.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:31 smithi049 ceph-mon[32331]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 587 op/s 2024-03-26T18:04:33.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:33 smithi171 ceph-mon[36033]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.0 MiB/s wr, 674 op/s 2024-03-26T18:04:33.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:33 smithi115 ceph-mon[36174]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.0 MiB/s wr, 674 op/s 2024-03-26T18:04:33.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:33 smithi049 ceph-mon[32331]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.0 MiB/s wr, 674 op/s 2024-03-26T18:04:35.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 130.0 s, 2246.6 tps, lat 44.566 ms stddev 54.921 2024-03-26T18:04:35.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:35 smithi171 ceph-mon[36033]: pgmap v77: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.4 MiB/s wr, 758 op/s 2024-03-26T18:04:35.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:35 smithi115 ceph-mon[36174]: pgmap v77: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.4 MiB/s wr, 758 op/s 2024-03-26T18:04:35.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:35 smithi049 ceph-mon[32331]: pgmap v77: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.4 MiB/s wr, 758 op/s 2024-03-26T18:04:37.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:37 smithi171 ceph-mon[36033]: pgmap v78: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 524 op/s 2024-03-26T18:04:37.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:37 smithi115 ceph-mon[36174]: pgmap v78: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 524 op/s 2024-03-26T18:04:37.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:37 smithi049 ceph-mon[32331]: pgmap v78: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 524 op/s 2024-03-26T18:04:39.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:39 smithi171 ceph-mon[36033]: pgmap v79: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-03-26T18:04:39.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:39 smithi115 ceph-mon[36174]: pgmap v79: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-03-26T18:04:39.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:39 smithi049 ceph-mon[32331]: pgmap v79: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-03-26T18:04:40.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 135.0 s, 1641.8 tps, lat 60.220 ms stddev 64.465 2024-03-26T18:04:41.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:41 smithi171 ceph-mon[36033]: pgmap v80: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 MiB/s wr, 565 op/s 2024-03-26T18:04:41.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:41 smithi115 ceph-mon[36174]: pgmap v80: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 MiB/s wr, 565 op/s 2024-03-26T18:04:41.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:41 smithi049 ceph-mon[32331]: pgmap v80: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 MiB/s wr, 565 op/s 2024-03-26T18:04:43.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:43 smithi171 ceph-mon[36033]: pgmap v81: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 607 op/s 2024-03-26T18:04:43.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:43 smithi115 ceph-mon[36174]: pgmap v81: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 607 op/s 2024-03-26T18:04:43.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:43 smithi049 ceph-mon[32331]: pgmap v81: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 607 op/s 2024-03-26T18:04:45.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 140.0 s, 1211.6 tps, lat 82.512 ms stddev 114.425 2024-03-26T18:04:45.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:45 smithi171 ceph-mon[36033]: pgmap v82: 129 pgs: 129 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 983 op/s 2024-03-26T18:04:45.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:45 smithi115 ceph-mon[36174]: pgmap v82: 129 pgs: 129 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 983 op/s 2024-03-26T18:04:45.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:45 smithi049 ceph-mon[32331]: pgmap v82: 129 pgs: 129 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 983 op/s 2024-03-26T18:04:47.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:47 smithi171 ceph-mon[36033]: pgmap v83: 129 pgs: 129 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 9.0 MiB/s wr, 806 op/s 2024-03-26T18:04:47.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:47 smithi115 ceph-mon[36174]: pgmap v83: 129 pgs: 129 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 9.0 MiB/s wr, 806 op/s 2024-03-26T18:04:47.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:47 smithi049 ceph-mon[32331]: pgmap v83: 129 pgs: 129 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 9.0 MiB/s wr, 806 op/s 2024-03-26T18:04:49.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:49 smithi171 ceph-mon[36033]: pgmap v84: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-03-26T18:04:49.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:49 smithi115 ceph-mon[36174]: pgmap v84: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-03-26T18:04:49.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:49 smithi049 ceph-mon[32331]: pgmap v84: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-03-26T18:04:50.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 145.0 s, 2187.4 tps, lat 46.543 ms stddev 55.437 2024-03-26T18:04:51.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:51 smithi115 ceph-mon[36174]: pgmap v85: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 881 op/s 2024-03-26T18:04:51.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:51 smithi115 ceph-mon[36174]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:04:51.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:51 smithi049 ceph-mon[32331]: pgmap v85: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 881 op/s 2024-03-26T18:04:51.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:51 smithi049 ceph-mon[32331]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:04:51.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:51 smithi171 ceph-mon[36033]: pgmap v85: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 881 op/s 2024-03-26T18:04:51.604 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:51 smithi171 ceph-mon[36033]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:04:53.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:53 smithi115 ceph-mon[36174]: pgmap v86: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-03-26T18:04:53.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:53 smithi049 ceph-mon[32331]: pgmap v86: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-03-26T18:04:53.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:53 smithi171 ceph-mon[36033]: pgmap v86: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-03-26T18:04:55.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 150.0 s, 2007.0 tps, lat 49.597 ms stddev 60.728 2024-03-26T18:04:55.404 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:55 smithi171 ceph-mon[36033]: pgmap v87: 129 pgs: 129 active+clean; 1.7 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2024-03-26T18:04:55.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:55 smithi115 ceph-mon[36174]: pgmap v87: 129 pgs: 129 active+clean; 1.7 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2024-03-26T18:04:55.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:55 smithi049 ceph-mon[32331]: pgmap v87: 129 pgs: 129 active+clean; 1.7 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2024-03-26T18:04:57.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:57 smithi115 ceph-mon[36174]: pgmap v88: 129 pgs: 129 active+clean; 1.7 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.7 MiB/s wr, 688 op/s 2024-03-26T18:04:57.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:57 smithi049 ceph-mon[32331]: pgmap v88: 129 pgs: 129 active+clean; 1.7 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.7 MiB/s wr, 688 op/s 2024-03-26T18:04:57.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:57 smithi171 ceph-mon[36033]: pgmap v88: 129 pgs: 129 active+clean; 1.7 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.7 MiB/s wr, 688 op/s 2024-03-26T18:04:59.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:04:59 smithi115 ceph-mon[36174]: pgmap v89: 129 pgs: 129 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 947 op/s 2024-03-26T18:04:59.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:04:59 smithi049 ceph-mon[32331]: pgmap v89: 129 pgs: 129 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 947 op/s 2024-03-26T18:04:59.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:04:59 smithi171 ceph-mon[36033]: pgmap v89: 129 pgs: 129 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 947 op/s 2024-03-26T18:05:00.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 155.0 s, 1650.8 tps, lat 59.672 ms stddev 64.738 2024-03-26T18:05:00.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:00 smithi115 ceph-mon[36174]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:05:00.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:00 smithi049 ceph-mon[32331]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:05:00.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:00 smithi171 ceph-mon[36033]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:05:01.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:01 smithi115 ceph-mon[36174]: pgmap v90: 129 pgs: 129 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 MiB/s wr, 570 op/s 2024-03-26T18:05:01.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:01 smithi049 ceph-mon[32331]: pgmap v90: 129 pgs: 129 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 MiB/s wr, 570 op/s 2024-03-26T18:05:01.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:01 smithi171 ceph-mon[36033]: pgmap v90: 129 pgs: 129 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 MiB/s wr, 570 op/s 2024-03-26T18:05:03.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:03 smithi115 ceph-mon[36174]: pgmap v91: 129 pgs: 129 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 593 op/s 2024-03-26T18:05:03.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:03 smithi049 ceph-mon[32331]: pgmap v91: 129 pgs: 129 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 593 op/s 2024-03-26T18:05:03.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:03 smithi171 ceph-mon[36033]: pgmap v91: 129 pgs: 129 active+clean; 1.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 593 op/s 2024-03-26T18:05:05.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 160.0 s, 280.4 tps, lat 345.792 ms stddev 516.496 2024-03-26T18:05:05.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:05 smithi115 ceph-mon[36174]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 722 op/s 2024-03-26T18:05:05.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:05 smithi049 ceph-mon[32331]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 722 op/s 2024-03-26T18:05:05.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:05 smithi171 ceph-mon[36033]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 722 op/s 2024-03-26T18:05:07.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:07 smithi115 ceph-mon[36174]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-03-26T18:05:07.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:07 smithi049 ceph-mon[32331]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-03-26T18:05:07.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:07 smithi171 ceph-mon[36033]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-03-26T18:05:09.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:09 smithi115 ceph-mon[36174]: pgmap v94: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 23 MiB/s wr, 1.72k op/s 2024-03-26T18:05:09.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:09 smithi049 ceph-mon[32331]: pgmap v94: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 23 MiB/s wr, 1.72k op/s 2024-03-26T18:05:09.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:09 smithi171 ceph-mon[36033]: pgmap v94: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 23 MiB/s wr, 1.72k op/s 2024-03-26T18:05:10.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 165.0 s, 1785.0 tps, lat 58.643 ms stddev 81.045 2024-03-26T18:05:10.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:10 smithi115 ceph-mon[36174]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:05:10.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:10 smithi049 ceph-mon[32331]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:05:10.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:10 smithi171 ceph-mon[36033]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:05:11.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:11 smithi115 ceph-mon[36174]: pgmap v95: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.46k op/s 2024-03-26T18:05:11.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:11 smithi049 ceph-mon[32331]: pgmap v95: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.46k op/s 2024-03-26T18:05:11.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:11 smithi171 ceph-mon[36033]: pgmap v95: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.46k op/s 2024-03-26T18:05:13.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:13 smithi115 ceph-mon[36174]: pgmap v96: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-03-26T18:05:13.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:13 smithi049 ceph-mon[32331]: pgmap v96: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-03-26T18:05:13.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:13 smithi171 ceph-mon[36033]: pgmap v96: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-03-26T18:05:15.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 170.0 s, 1539.2 tps, lat 63.947 ms stddev 75.231 2024-03-26T18:05:15.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:15 smithi115 ceph-mon[36174]: pgmap v97: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 23 MiB/s wr, 1.69k op/s 2024-03-26T18:05:15.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:15 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:05:15.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:15 smithi049 ceph-mon[32331]: pgmap v97: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 23 MiB/s wr, 1.69k op/s 2024-03-26T18:05:15.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:15 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:05:15.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:15 smithi171 ceph-mon[36033]: pgmap v97: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 23 MiB/s wr, 1.69k op/s 2024-03-26T18:05:15.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:15 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:05:16.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:16 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:05:16.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:16 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:05:16.485 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:16 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:05:16.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:16 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:05:16.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:16 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:05:16.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:16 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:05:16.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:16 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:05:16.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:16 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:05:16.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:16 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:05:17.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:17 smithi115 ceph-mon[36174]: pgmap v98: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 1.49k op/s 2024-03-26T18:05:17.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:17 smithi049 ceph-mon[32331]: pgmap v98: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 1.49k op/s 2024-03-26T18:05:17.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:17 smithi171 ceph-mon[36033]: pgmap v98: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 1.49k op/s 2024-03-26T18:05:19.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:19 smithi115 ceph-mon[36174]: pgmap v99: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 MiB/s wr, 1.76k op/s 2024-03-26T18:05:19.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:19 smithi049 ceph-mon[32331]: pgmap v99: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 MiB/s wr, 1.76k op/s 2024-03-26T18:05:19.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:19 smithi171 ceph-mon[36033]: pgmap v99: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 MiB/s wr, 1.76k op/s 2024-03-26T18:05:20.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 175.0 s, 1851.0 tps, lat 54.783 ms stddev 63.715 2024-03-26T18:05:21.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:21 smithi115 ceph-mon[36174]: pgmap v100: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 528 op/s 2024-03-26T18:05:21.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:21 smithi049 ceph-mon[32331]: pgmap v100: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 528 op/s 2024-03-26T18:05:21.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:21 smithi171 ceph-mon[36033]: pgmap v100: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 528 op/s 2024-03-26T18:05:23.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:23 smithi115 ceph-mon[36174]: pgmap v101: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 616 op/s 2024-03-26T18:05:23.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:23 smithi049 ceph-mon[32331]: pgmap v101: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 616 op/s 2024-03-26T18:05:23.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:23 smithi171 ceph-mon[36033]: pgmap v101: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 616 op/s 2024-03-26T18:05:25.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 180.0 s, 1647.2 tps, lat 60.183 ms stddev 72.423 2024-03-26T18:05:25.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:25 smithi115 ceph-mon[36174]: pgmap v102: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.2 MiB/s wr, 759 op/s 2024-03-26T18:05:25.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:25 smithi049 ceph-mon[32331]: pgmap v102: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.2 MiB/s wr, 759 op/s 2024-03-26T18:05:25.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:25 smithi171 ceph-mon[36033]: pgmap v102: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.2 MiB/s wr, 759 op/s 2024-03-26T18:05:27.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:27 smithi115 ceph-mon[36174]: pgmap v103: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.1 MiB/s wr, 531 op/s 2024-03-26T18:05:27.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:27 smithi049 ceph-mon[32331]: pgmap v103: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.1 MiB/s wr, 531 op/s 2024-03-26T18:05:27.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:27 smithi171 ceph-mon[36033]: pgmap v103: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.1 MiB/s wr, 531 op/s 2024-03-26T18:05:29.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:29 smithi115 ceph-mon[36174]: pgmap v104: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 821 op/s 2024-03-26T18:05:29.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:29 smithi049 ceph-mon[32331]: pgmap v104: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 821 op/s 2024-03-26T18:05:29.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:29 smithi171 ceph-mon[36033]: pgmap v104: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 821 op/s 2024-03-26T18:05:30.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 185.0 s, 1761.4 tps, lat 57.082 ms stddev 80.475 2024-03-26T18:05:31.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:31 smithi115 ceph-mon[36174]: pgmap v105: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 547 op/s 2024-03-26T18:05:31.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:31 smithi049 ceph-mon[32331]: pgmap v105: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 547 op/s 2024-03-26T18:05:31.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:31 smithi171 ceph-mon[36033]: pgmap v105: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 547 op/s 2024-03-26T18:05:33.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:33 smithi115 ceph-mon[36174]: pgmap v106: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 615 op/s 2024-03-26T18:05:33.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:33 smithi049 ceph-mon[32331]: pgmap v106: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 615 op/s 2024-03-26T18:05:33.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:33 smithi171 ceph-mon[36033]: pgmap v106: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 615 op/s 2024-03-26T18:05:35.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 190.0 s, 1439.6 tps, lat 68.609 ms stddev 84.364 2024-03-26T18:05:35.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:35 smithi115 ceph-mon[36174]: pgmap v107: 129 pgs: 129 active+clean; 2.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 838 op/s 2024-03-26T18:05:35.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:35 smithi049 ceph-mon[32331]: pgmap v107: 129 pgs: 129 active+clean; 2.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 838 op/s 2024-03-26T18:05:35.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:35 smithi171 ceph-mon[36033]: pgmap v107: 129 pgs: 129 active+clean; 2.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.1 MiB/s wr, 838 op/s 2024-03-26T18:05:37.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:37 smithi115 ceph-mon[36174]: pgmap v108: 129 pgs: 129 active+clean; 2.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 669 op/s 2024-03-26T18:05:37.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:37 smithi049 ceph-mon[32331]: pgmap v108: 129 pgs: 129 active+clean; 2.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 669 op/s 2024-03-26T18:05:37.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:37 smithi171 ceph-mon[36033]: pgmap v108: 129 pgs: 129 active+clean; 2.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 669 op/s 2024-03-26T18:05:39.484 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:39 smithi115 ceph-mon[36174]: pgmap v109: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 919 op/s 2024-03-26T18:05:39.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:39 smithi049 ceph-mon[32331]: pgmap v109: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 919 op/s 2024-03-26T18:05:39.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:39 smithi171 ceph-mon[36033]: pgmap v109: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 919 op/s 2024-03-26T18:05:40.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 195.0 s, 1709.8 tps, lat 59.112 ms stddev 68.697 2024-03-26T18:05:41.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:41 smithi049 ceph-mon[32331]: pgmap v110: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.6 MiB/s wr, 629 op/s 2024-03-26T18:05:41.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:41 smithi171 ceph-mon[36033]: pgmap v110: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.6 MiB/s wr, 629 op/s 2024-03-26T18:05:41.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:41 smithi115 ceph-mon[36174]: pgmap v110: 129 pgs: 129 active+clean; 2.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.6 MiB/s wr, 629 op/s 2024-03-26T18:05:43.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:43 smithi049 ceph-mon[32331]: pgmap v111: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.3 MiB/s wr, 697 op/s 2024-03-26T18:05:43.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:43 smithi171 ceph-mon[36033]: pgmap v111: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.3 MiB/s wr, 697 op/s 2024-03-26T18:05:43.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:43 smithi115 ceph-mon[36174]: pgmap v111: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.3 MiB/s wr, 697 op/s 2024-03-26T18:05:45.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 200.0 s, 1396.2 tps, lat 71.393 ms stddev 103.904 2024-03-26T18:05:45.499 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:45 smithi049 ceph-mon[32331]: pgmap v112: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 815 op/s 2024-03-26T18:05:45.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:45 smithi171 ceph-mon[36033]: pgmap v112: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 815 op/s 2024-03-26T18:05:45.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:45 smithi115 ceph-mon[36174]: pgmap v112: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 815 op/s 2024-03-26T18:05:47.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:47 smithi171 ceph-mon[36033]: pgmap v113: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 504 op/s 2024-03-26T18:05:47.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:47 smithi115 ceph-mon[36174]: pgmap v113: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 504 op/s 2024-03-26T18:05:47.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:47 smithi049 ceph-mon[32331]: pgmap v113: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 504 op/s 2024-03-26T18:05:49.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:49 smithi171 ceph-mon[36033]: pgmap v114: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2024-03-26T18:05:49.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:49 smithi115 ceph-mon[36174]: pgmap v114: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2024-03-26T18:05:49.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:49 smithi049 ceph-mon[32331]: pgmap v114: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2024-03-26T18:05:50.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 205.0 s, 1882.2 tps, lat 53.417 ms stddev 67.065 2024-03-26T18:05:51.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:51 smithi171 ceph-mon[36033]: pgmap v115: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 832 op/s 2024-03-26T18:05:51.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:51 smithi115 ceph-mon[36174]: pgmap v115: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 832 op/s 2024-03-26T18:05:51.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:51 smithi049 ceph-mon[32331]: pgmap v115: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 832 op/s 2024-03-26T18:05:53.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:53 smithi171 ceph-mon[36033]: pgmap v116: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 915 op/s 2024-03-26T18:05:53.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:53 smithi115 ceph-mon[36174]: pgmap v116: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 915 op/s 2024-03-26T18:05:53.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:53 smithi049 ceph-mon[32331]: pgmap v116: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 915 op/s 2024-03-26T18:05:55.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 210.0 s, 1356.8 tps, lat 73.038 ms stddev 85.481 2024-03-26T18:05:55.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:55 smithi171 ceph-mon[36033]: pgmap v117: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-03-26T18:05:55.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:55 smithi115 ceph-mon[36174]: pgmap v117: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-03-26T18:05:55.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:55 smithi049 ceph-mon[32331]: pgmap v117: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-03-26T18:05:57.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:57 smithi171 ceph-mon[36033]: pgmap v118: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 842 op/s 2024-03-26T18:05:57.735 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:57 smithi115 ceph-mon[36174]: pgmap v118: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 842 op/s 2024-03-26T18:05:57.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:57 smithi049 ceph-mon[32331]: pgmap v118: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 842 op/s 2024-03-26T18:05:59.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:05:59 smithi171 ceph-mon[36033]: pgmap v119: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.10k op/s 2024-03-26T18:05:59.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:05:59 smithi115 ceph-mon[36174]: pgmap v119: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.10k op/s 2024-03-26T18:05:59.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:05:59 smithi049 ceph-mon[32331]: pgmap v119: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 1.10k op/s 2024-03-26T18:06:00.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 215.0 s, 1047.0 tps, lat 96.329 ms stddev 113.685 2024-03-26T18:06:01.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:01 smithi171 ceph-mon[36033]: pgmap v120: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 MiB/s wr, 526 op/s 2024-03-26T18:06:01.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:01 smithi115 ceph-mon[36174]: pgmap v120: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 MiB/s wr, 526 op/s 2024-03-26T18:06:01.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:01 smithi049 ceph-mon[32331]: pgmap v120: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 MiB/s wr, 526 op/s 2024-03-26T18:06:03.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:03 smithi171 ceph-mon[36033]: pgmap v121: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.0 MiB/s wr, 581 op/s 2024-03-26T18:06:03.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:03 smithi115 ceph-mon[36174]: pgmap v121: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.0 MiB/s wr, 581 op/s 2024-03-26T18:06:03.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:03 smithi049 ceph-mon[32331]: pgmap v121: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.0 MiB/s wr, 581 op/s 2024-03-26T18:06:05.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 220.0 s, 238.4 tps, lat 394.606 ms stddev 260.052 2024-03-26T18:06:05.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:05 smithi171 ceph-mon[36033]: pgmap v122: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 794 op/s 2024-03-26T18:06:05.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:05 smithi115 ceph-mon[36174]: pgmap v122: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 794 op/s 2024-03-26T18:06:05.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:05 smithi049 ceph-mon[32331]: pgmap v122: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 794 op/s 2024-03-26T18:06:07.550 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:07 smithi049 ceph-mon[32331]: pgmap v123: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.7 MiB/s wr, 614 op/s 2024-03-26T18:06:07.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:07 smithi171 ceph-mon[36033]: pgmap v123: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.7 MiB/s wr, 614 op/s 2024-03-26T18:06:07.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:07 smithi115 ceph-mon[36174]: pgmap v123: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.7 MiB/s wr, 614 op/s 2024-03-26T18:06:09.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:09 smithi171 ceph-mon[36033]: pgmap v124: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 MiB/s wr, 926 op/s 2024-03-26T18:06:09.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:09 smithi115 ceph-mon[36174]: pgmap v124: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 MiB/s wr, 926 op/s 2024-03-26T18:06:09.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:09 smithi049 ceph-mon[32331]: pgmap v124: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 MiB/s wr, 926 op/s 2024-03-26T18:06:10.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 225.0 s, 372.0 tps, lat 281.546 ms stddev 423.593 2024-03-26T18:06:11.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:11 smithi171 ceph-mon[36033]: pgmap v125: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-03-26T18:06:11.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:11 smithi115 ceph-mon[36174]: pgmap v125: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-03-26T18:06:11.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:11 smithi049 ceph-mon[32331]: pgmap v125: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-03-26T18:06:13.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:13 smithi171 ceph-mon[36033]: pgmap v126: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 897 op/s 2024-03-26T18:06:13.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:13 smithi115 ceph-mon[36174]: pgmap v126: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 897 op/s 2024-03-26T18:06:13.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:13 smithi049 ceph-mon[32331]: pgmap v126: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 897 op/s 2024-03-26T18:06:15.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 230.0 s, 1339.2 tps, lat 74.989 ms stddev 86.442 2024-03-26T18:06:15.602 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:15 smithi171 ceph-mon[36033]: pgmap v127: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-03-26T18:06:15.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:15 smithi115 ceph-mon[36174]: pgmap v127: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-03-26T18:06:15.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:15 smithi049 ceph-mon[32331]: pgmap v127: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-03-26T18:06:16.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:16 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:06:16.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:16 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:06:16.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:16 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:06:17.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:17 smithi171 ceph-mon[36033]: pgmap v128: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-03-26T18:06:17.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:17 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:06:17.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:17 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:06:17.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:17 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:06:17.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:17 smithi115 ceph-mon[36174]: pgmap v128: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-03-26T18:06:17.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:17 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:06:17.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:17 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:06:17.734 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:17 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:06:17.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:17 smithi049 ceph-mon[32331]: pgmap v128: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-03-26T18:06:17.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:17 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:06:17.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:17 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:06:17.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:17 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:06:19.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:19 smithi171 ceph-mon[36033]: pgmap v129: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2024-03-26T18:06:19.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:19 smithi115 ceph-mon[36174]: pgmap v129: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2024-03-26T18:06:19.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:19 smithi049 ceph-mon[32331]: pgmap v129: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2024-03-26T18:06:20.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 235.0 s, 896.4 tps, lat 111.168 ms stddev 115.091 2024-03-26T18:06:21.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:21 smithi171 ceph-mon[36033]: pgmap v130: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2024-03-26T18:06:21.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:21 smithi115 ceph-mon[36174]: pgmap v130: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2024-03-26T18:06:21.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:21 smithi049 ceph-mon[32331]: pgmap v130: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2024-03-26T18:06:23.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:23 smithi171 ceph-mon[36033]: pgmap v131: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 933 op/s 2024-03-26T18:06:23.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:23 smithi115 ceph-mon[36174]: pgmap v131: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 933 op/s 2024-03-26T18:06:23.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:23 smithi049 ceph-mon[32331]: pgmap v131: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 933 op/s 2024-03-26T18:06:25.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 240.0 s, 949.6 tps, lat 105.449 ms stddev 120.900 2024-03-26T18:06:25.603 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:25 smithi171 ceph-mon[36033]: pgmap v132: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-03-26T18:06:25.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:25 smithi115 ceph-mon[36174]: pgmap v132: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-03-26T18:06:25.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:25 smithi049 ceph-mon[32331]: pgmap v132: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-03-26T18:06:27.602 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:27 smithi171 ceph-mon[36033]: pgmap v133: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.9 MiB/s wr, 466 op/s 2024-03-26T18:06:27.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:27 smithi115 ceph-mon[36174]: pgmap v133: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.9 MiB/s wr, 466 op/s 2024-03-26T18:06:27.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:27 smithi049 ceph-mon[32331]: pgmap v133: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.9 MiB/s wr, 466 op/s 2024-03-26T18:06:29.602 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:29 smithi171 ceph-mon[36033]: pgmap v134: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.0 MiB/s wr, 700 op/s 2024-03-26T18:06:29.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:29 smithi115 ceph-mon[36174]: pgmap v134: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.0 MiB/s wr, 700 op/s 2024-03-26T18:06:29.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:29 smithi049 ceph-mon[32331]: pgmap v134: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.0 MiB/s wr, 700 op/s 2024-03-26T18:06:30.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 245.0 s, 1178.2 tps, lat 84.820 ms stddev 98.702 2024-03-26T18:06:31.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:31 smithi115 ceph-mon[36174]: pgmap v135: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-03-26T18:06:31.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:31 smithi049 ceph-mon[32331]: pgmap v135: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-03-26T18:06:31.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:31 smithi171 ceph-mon[36033]: pgmap v135: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-03-26T18:06:33.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:33 smithi115 ceph-mon[36174]: pgmap v136: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.5 MiB/s wr, 521 op/s 2024-03-26T18:06:33.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:33 smithi049 ceph-mon[32331]: pgmap v136: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.5 MiB/s wr, 521 op/s 2024-03-26T18:06:33.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:33 smithi171 ceph-mon[36033]: pgmap v136: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.5 MiB/s wr, 521 op/s 2024-03-26T18:06:35.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 250.0 s, 1562.8 tps, lat 64.132 ms stddev 79.170 2024-03-26T18:06:35.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:35 smithi115 ceph-mon[36174]: pgmap v137: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.1 MiB/s wr, 782 op/s 2024-03-26T18:06:35.749 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:35 smithi049 ceph-mon[32331]: pgmap v137: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.1 MiB/s wr, 782 op/s 2024-03-26T18:06:35.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:35 smithi171 ceph-mon[36033]: pgmap v137: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.1 MiB/s wr, 782 op/s 2024-03-26T18:06:37.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:37 smithi115 ceph-mon[36174]: pgmap v138: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 565 op/s 2024-03-26T18:06:37.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:37 smithi049 ceph-mon[32331]: pgmap v138: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 565 op/s 2024-03-26T18:06:37.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:37 smithi171 ceph-mon[36033]: pgmap v138: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 565 op/s 2024-03-26T18:06:39.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:39 smithi115 ceph-mon[36174]: pgmap v139: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 700 op/s 2024-03-26T18:06:39.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:39 smithi049 ceph-mon[32331]: pgmap v139: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 700 op/s 2024-03-26T18:06:39.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:39 smithi171 ceph-mon[36033]: pgmap v139: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.6 MiB/s wr, 700 op/s 2024-03-26T18:06:40.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 255.0 s, 1122.2 tps, lat 89.759 ms stddev 115.884 2024-03-26T18:06:41.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:41 smithi115 ceph-mon[36174]: pgmap v140: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 MiB/s wr, 581 op/s 2024-03-26T18:06:41.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:41 smithi049 ceph-mon[32331]: pgmap v140: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 MiB/s wr, 581 op/s 2024-03-26T18:06:41.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:41 smithi171 ceph-mon[36033]: pgmap v140: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 MiB/s wr, 581 op/s 2024-03-26T18:06:43.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:43 smithi115 ceph-mon[36174]: pgmap v141: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-03-26T18:06:43.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:43 smithi049 ceph-mon[32331]: pgmap v141: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-03-26T18:06:43.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:43 smithi171 ceph-mon[36033]: pgmap v141: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-03-26T18:06:45.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 260.0 s, 834.8 tps, lat 118.276 ms stddev 135.847 2024-03-26T18:06:45.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:45 smithi115 ceph-mon[36174]: pgmap v142: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 754 op/s 2024-03-26T18:06:45.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:45 smithi049 ceph-mon[32331]: pgmap v142: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 754 op/s 2024-03-26T18:06:45.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:45 smithi171 ceph-mon[36033]: pgmap v142: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 754 op/s 2024-03-26T18:06:47.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:47 smithi115 ceph-mon[36174]: pgmap v143: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 480 op/s 2024-03-26T18:06:47.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:47 smithi049 ceph-mon[32331]: pgmap v143: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 480 op/s 2024-03-26T18:06:47.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:47 smithi171 ceph-mon[36033]: pgmap v143: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 480 op/s 2024-03-26T18:06:49.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:49 smithi115 ceph-mon[36174]: pgmap v144: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 619 op/s 2024-03-26T18:06:49.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:49 smithi049 ceph-mon[32331]: pgmap v144: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 619 op/s 2024-03-26T18:06:49.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:49 smithi171 ceph-mon[36033]: pgmap v144: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 619 op/s 2024-03-26T18:06:50.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 265.0 s, 997.2 tps, lat 101.112 ms stddev 118.153 2024-03-26T18:06:51.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:51 smithi115 ceph-mon[36174]: pgmap v145: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.1 MiB/s wr, 576 op/s 2024-03-26T18:06:51.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:51 smithi049 ceph-mon[32331]: pgmap v145: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.1 MiB/s wr, 576 op/s 2024-03-26T18:06:51.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:51 smithi171 ceph-mon[36033]: pgmap v145: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.1 MiB/s wr, 576 op/s 2024-03-26T18:06:53.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:53 smithi115 ceph-mon[36174]: pgmap v146: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 473 op/s 2024-03-26T18:06:53.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:53 smithi049 ceph-mon[32331]: pgmap v146: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 473 op/s 2024-03-26T18:06:53.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:53 smithi171 ceph-mon[36033]: pgmap v146: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 473 op/s 2024-03-26T18:06:55.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 270.0 s, 1109.6 tps, lat 89.560 ms stddev 98.400 2024-03-26T18:06:55.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:55 smithi115 ceph-mon[36174]: pgmap v147: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.9 MiB/s wr, 687 op/s 2024-03-26T18:06:55.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:55 smithi049 ceph-mon[32331]: pgmap v147: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.9 MiB/s wr, 687 op/s 2024-03-26T18:06:55.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:55 smithi171 ceph-mon[36033]: pgmap v147: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.9 MiB/s wr, 687 op/s 2024-03-26T18:06:57.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:57 smithi115 ceph-mon[36174]: pgmap v148: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-03-26T18:06:57.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:57 smithi049 ceph-mon[32331]: pgmap v148: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-03-26T18:06:57.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:57 smithi171 ceph-mon[36033]: pgmap v148: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-03-26T18:06:59.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:06:59 smithi115 ceph-mon[36174]: pgmap v149: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 642 op/s 2024-03-26T18:06:59.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:06:59 smithi049 ceph-mon[32331]: pgmap v149: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 642 op/s 2024-03-26T18:06:59.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:06:59 smithi171 ceph-mon[36033]: pgmap v149: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 642 op/s 2024-03-26T18:07:00.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 275.0 s, 746.2 tps, lat 127.488 ms stddev 136.916 2024-03-26T18:07:01.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:01 smithi115 ceph-mon[36174]: pgmap v150: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 595 op/s 2024-03-26T18:07:01.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:01 smithi049 ceph-mon[32331]: pgmap v150: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 595 op/s 2024-03-26T18:07:01.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:01 smithi171 ceph-mon[36033]: pgmap v150: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 595 op/s 2024-03-26T18:07:03.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:03 smithi115 ceph-mon[36174]: pgmap v151: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.4 MiB/s wr, 503 op/s 2024-03-26T18:07:03.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:03 smithi049 ceph-mon[32331]: pgmap v151: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.4 MiB/s wr, 503 op/s 2024-03-26T18:07:03.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:03 smithi171 ceph-mon[36033]: pgmap v151: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.4 MiB/s wr, 503 op/s 2024-03-26T18:07:05.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 280.0 s, 253.6 tps, lat 398.409 ms stddev 552.963 2024-03-26T18:07:05.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:05 smithi115 ceph-mon[36174]: pgmap v152: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-03-26T18:07:05.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:05 smithi049 ceph-mon[32331]: pgmap v152: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-03-26T18:07:05.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:05 smithi171 ceph-mon[36033]: pgmap v152: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-03-26T18:07:07.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:07 smithi115 ceph-mon[36174]: pgmap v153: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 1.15k op/s 2024-03-26T18:07:07.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:07 smithi049 ceph-mon[32331]: pgmap v153: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 1.15k op/s 2024-03-26T18:07:07.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:07 smithi171 ceph-mon[36033]: pgmap v153: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 1.15k op/s 2024-03-26T18:07:09.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:09 smithi115 ceph-mon[36174]: pgmap v154: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-03-26T18:07:09.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:09 smithi049 ceph-mon[32331]: pgmap v154: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-03-26T18:07:09.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:09 smithi171 ceph-mon[36033]: pgmap v154: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 1.38k op/s 2024-03-26T18:07:10.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 285.0 s, 692.2 tps, lat 148.268 ms stddev 186.202 2024-03-26T18:07:11.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:11 smithi115 ceph-mon[36174]: pgmap v155: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 1.41k op/s 2024-03-26T18:07:11.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:11 smithi049 ceph-mon[32331]: pgmap v155: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 1.41k op/s 2024-03-26T18:07:11.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:11 smithi171 ceph-mon[36033]: pgmap v155: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 1.41k op/s 2024-03-26T18:07:13.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:13 smithi115 ceph-mon[36174]: pgmap v156: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-03-26T18:07:13.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:13 smithi049 ceph-mon[32331]: pgmap v156: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-03-26T18:07:13.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:13 smithi171 ceph-mon[36033]: pgmap v156: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-03-26T18:07:15.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 290.0 s, 867.6 tps, lat 117.677 ms stddev 133.026 2024-03-26T18:07:15.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:15 smithi115 ceph-mon[36174]: pgmap v157: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-03-26T18:07:15.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:15 smithi049 ceph-mon[32331]: pgmap v157: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-03-26T18:07:15.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:15 smithi171 ceph-mon[36033]: pgmap v157: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-03-26T18:07:17.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:17 smithi115 ceph-mon[36174]: pgmap v158: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 663 op/s 2024-03-26T18:07:17.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:17 smithi049 ceph-mon[32331]: pgmap v158: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 663 op/s 2024-03-26T18:07:17.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:17 smithi171 ceph-mon[36033]: pgmap v158: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 663 op/s 2024-03-26T18:07:18.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:18 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:07:18.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:18 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:07:18.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:18 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:07:19.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:19 smithi049 ceph-mon[32331]: pgmap v159: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.3 MiB/s wr, 777 op/s 2024-03-26T18:07:19.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:19 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:07:19.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:19 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:07:19.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:19 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:07:19.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:19 smithi171 ceph-mon[36033]: pgmap v159: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.3 MiB/s wr, 777 op/s 2024-03-26T18:07:19.853 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:19 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:07:19.853 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:19 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:07:19.853 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:19 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:07:19.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:19 smithi115 ceph-mon[36174]: pgmap v159: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.3 MiB/s wr, 777 op/s 2024-03-26T18:07:19.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:19 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:07:19.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:19 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:07:19.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:19 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:07:20.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 295.0 s, 785.8 tps, lat 126.487 ms stddev 142.246 2024-03-26T18:07:21.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:21 smithi049 ceph-mon[32331]: pgmap v160: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 657 op/s 2024-03-26T18:07:21.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:21 smithi171 ceph-mon[36033]: pgmap v160: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 657 op/s 2024-03-26T18:07:21.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:21 smithi115 ceph-mon[36174]: pgmap v160: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 657 op/s 2024-03-26T18:07:23.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:23 smithi049 ceph-mon[32331]: pgmap v161: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.9 MiB/s wr, 460 op/s 2024-03-26T18:07:23.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:23 smithi171 ceph-mon[36033]: pgmap v161: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.9 MiB/s wr, 460 op/s 2024-03-26T18:07:23.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:23 smithi115 ceph-mon[36174]: pgmap v161: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.9 MiB/s wr, 460 op/s 2024-03-26T18:07:25.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 300.0 s, 675.0 tps, lat 147.500 ms stddev 180.712 2024-03-26T18:07:25.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:25 smithi049 ceph-mon[32331]: pgmap v162: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-03-26T18:07:25.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:25 smithi171 ceph-mon[36033]: pgmap v162: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-03-26T18:07:25.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:25 smithi115 ceph-mon[36174]: pgmap v162: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-03-26T18:07:27.748 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:27 smithi049 ceph-mon[32331]: pgmap v163: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 484 op/s 2024-03-26T18:07:27.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:27 smithi171 ceph-mon[36033]: pgmap v163: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 484 op/s 2024-03-26T18:07:27.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:27 smithi115 ceph-mon[36174]: pgmap v163: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 484 op/s 2024-03-26T18:07:29.853 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:29 smithi171 ceph-mon[36033]: pgmap v164: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 618 op/s 2024-03-26T18:07:29.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:29 smithi115 ceph-mon[36174]: pgmap v164: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 618 op/s 2024-03-26T18:07:29.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:29 smithi049 ceph-mon[32331]: pgmap v164: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 618 op/s 2024-03-26T18:07:30.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 305.0 s, 737.2 tps, lat 134.597 ms stddev 166.489 2024-03-26T18:07:31.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:31 smithi171 ceph-mon[36033]: pgmap v165: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 733 op/s 2024-03-26T18:07:31.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:31 smithi115 ceph-mon[36174]: pgmap v165: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 733 op/s 2024-03-26T18:07:31.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:31 smithi049 ceph-mon[32331]: pgmap v165: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 733 op/s 2024-03-26T18:07:33.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:33 smithi171 ceph-mon[36033]: pgmap v166: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 616 op/s 2024-03-26T18:07:33.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:33 smithi115 ceph-mon[36174]: pgmap v166: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 616 op/s 2024-03-26T18:07:33.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:33 smithi049 ceph-mon[32331]: pgmap v166: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 616 op/s 2024-03-26T18:07:35.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 310.0 s, 824.4 tps, lat 123.045 ms stddev 162.387 2024-03-26T18:07:35.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:35 smithi171 ceph-mon[36033]: pgmap v167: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 20 MiB/s wr, 877 op/s 2024-03-26T18:07:35.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:35 smithi115 ceph-mon[36174]: pgmap v167: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 20 MiB/s wr, 877 op/s 2024-03-26T18:07:35.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:35 smithi049 ceph-mon[32331]: pgmap v167: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 20 MiB/s wr, 877 op/s 2024-03-26T18:07:37.800 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:37 smithi049 ceph-mon[32331]: pgmap v168: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 623 op/s 2024-03-26T18:07:37.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:37 smithi171 ceph-mon[36033]: pgmap v168: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 623 op/s 2024-03-26T18:07:37.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:37 smithi115 ceph-mon[36174]: pgmap v168: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 623 op/s 2024-03-26T18:07:39.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:39 smithi115 ceph-mon[36174]: pgmap v169: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 719 op/s 2024-03-26T18:07:39.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:39 smithi171 ceph-mon[36033]: pgmap v169: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 719 op/s 2024-03-26T18:07:39.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:39 smithi049 ceph-mon[32331]: pgmap v169: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 719 op/s 2024-03-26T18:07:40.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 315.0 s, 731.6 tps, lat 134.625 ms stddev 150.902 2024-03-26T18:07:41.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:41 smithi171 ceph-mon[36033]: pgmap v170: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 732 op/s 2024-03-26T18:07:41.986 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:41 smithi115 ceph-mon[36174]: pgmap v170: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 732 op/s 2024-03-26T18:07:41.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:41 smithi049 ceph-mon[32331]: pgmap v170: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 732 op/s 2024-03-26T18:07:43.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:43 smithi171 ceph-mon[36033]: pgmap v171: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 502 op/s 2024-03-26T18:07:43.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:43 smithi115 ceph-mon[36174]: pgmap v171: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 502 op/s 2024-03-26T18:07:43.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:43 smithi049 ceph-mon[32331]: pgmap v171: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 502 op/s 2024-03-26T18:07:45.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 320.0 s, 700.0 tps, lat 143.774 ms stddev 165.589 2024-03-26T18:07:45.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:45 smithi171 ceph-mon[36033]: pgmap v172: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 734 op/s 2024-03-26T18:07:45.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:45 smithi115 ceph-mon[36174]: pgmap v172: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 734 op/s 2024-03-26T18:07:45.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:45 smithi049 ceph-mon[32331]: pgmap v172: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 734 op/s 2024-03-26T18:07:47.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:47 smithi171 ceph-mon[36033]: pgmap v173: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 473 op/s 2024-03-26T18:07:47.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:47 smithi115 ceph-mon[36174]: pgmap v173: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 473 op/s 2024-03-26T18:07:47.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:47 smithi049 ceph-mon[32331]: pgmap v173: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 473 op/s 2024-03-26T18:07:49.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:49 smithi171 ceph-mon[36033]: pgmap v174: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 15 MiB/s wr, 549 op/s 2024-03-26T18:07:49.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:49 smithi115 ceph-mon[36174]: pgmap v174: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 15 MiB/s wr, 549 op/s 2024-03-26T18:07:49.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:49 smithi049 ceph-mon[32331]: pgmap v174: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 15 MiB/s wr, 549 op/s 2024-03-26T18:07:50.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 325.0 s, 841.2 tps, lat 117.871 ms stddev 154.531 2024-03-26T18:07:51.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:51 smithi171 ceph-mon[36033]: pgmap v175: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 17 MiB/s wr, 611 op/s 2024-03-26T18:07:51.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:51 smithi115 ceph-mon[36174]: pgmap v175: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 17 MiB/s wr, 611 op/s 2024-03-26T18:07:51.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:51 smithi049 ceph-mon[32331]: pgmap v175: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 17 MiB/s wr, 611 op/s 2024-03-26T18:07:53.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:53 smithi171 ceph-mon[36033]: pgmap v176: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 12 MiB/s wr, 465 op/s 2024-03-26T18:07:53.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:53 smithi115 ceph-mon[36174]: pgmap v176: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 12 MiB/s wr, 465 op/s 2024-03-26T18:07:53.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:53 smithi049 ceph-mon[32331]: pgmap v176: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 12 MiB/s wr, 465 op/s 2024-03-26T18:07:55.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 330.0 s, 920.8 tps, lat 110.170 ms stddev 130.536 2024-03-26T18:07:55.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:55 smithi171 ceph-mon[36033]: pgmap v177: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 18 MiB/s wr, 701 op/s 2024-03-26T18:07:55.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:55 smithi115 ceph-mon[36174]: pgmap v177: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 18 MiB/s wr, 701 op/s 2024-03-26T18:07:55.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:55 smithi049 ceph-mon[32331]: pgmap v177: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 18 MiB/s wr, 701 op/s 2024-03-26T18:07:57.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:57 smithi171 ceph-mon[36033]: pgmap v178: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 13 MiB/s wr, 469 op/s 2024-03-26T18:07:57.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:57 smithi115 ceph-mon[36174]: pgmap v178: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 13 MiB/s wr, 469 op/s 2024-03-26T18:07:57.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:57 smithi049 ceph-mon[32331]: pgmap v178: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 13 MiB/s wr, 469 op/s 2024-03-26T18:07:59.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:07:59 smithi171 ceph-mon[36033]: pgmap v179: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 15 MiB/s wr, 568 op/s 2024-03-26T18:07:59.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:07:59 smithi115 ceph-mon[36174]: pgmap v179: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 15 MiB/s wr, 568 op/s 2024-03-26T18:07:59.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:07:59 smithi049 ceph-mon[32331]: pgmap v179: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 15 MiB/s wr, 568 op/s 2024-03-26T18:08:00.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 335.0 s, 827.4 tps, lat 120.316 ms stddev 128.030 2024-03-26T18:08:01.852 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:01 smithi171 ceph-mon[36033]: pgmap v180: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 17 MiB/s wr, 697 op/s 2024-03-26T18:08:01.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:01 smithi115 ceph-mon[36174]: pgmap v180: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 17 MiB/s wr, 697 op/s 2024-03-26T18:08:01.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:01 smithi049 ceph-mon[32331]: pgmap v180: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 17 MiB/s wr, 697 op/s 2024-03-26T18:08:03.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:03 smithi115 ceph-mon[36174]: pgmap v181: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 539 op/s 2024-03-26T18:08:03.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:03 smithi049 ceph-mon[32331]: pgmap v181: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 539 op/s 2024-03-26T18:08:04.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:03 smithi171 ceph-mon[36033]: pgmap v181: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 539 op/s 2024-03-26T18:08:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 340.0 s, 390.0 tps, lat 254.122 ms stddev 368.074 2024-03-26T18:08:05.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:05 smithi115 ceph-mon[36174]: pgmap v182: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2024-03-26T18:08:05.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:05 smithi049 ceph-mon[32331]: pgmap v182: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2024-03-26T18:08:06.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:05 smithi171 ceph-mon[36033]: pgmap v182: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2024-03-26T18:08:07.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:07 smithi115 ceph-mon[36174]: pgmap v183: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 13 MiB/s wr, 987 op/s 2024-03-26T18:08:07.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:07 smithi049 ceph-mon[32331]: pgmap v183: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 13 MiB/s wr, 987 op/s 2024-03-26T18:08:08.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:07 smithi171 ceph-mon[36033]: pgmap v183: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 13 MiB/s wr, 987 op/s 2024-03-26T18:08:09.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:09 smithi115 ceph-mon[36174]: pgmap v184: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-03-26T18:08:09.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:09 smithi049 ceph-mon[32331]: pgmap v184: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-03-26T18:08:10.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:09 smithi171 ceph-mon[36033]: pgmap v184: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-03-26T18:08:10.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 345.0 s, 816.8 tps, lat 123.615 ms stddev 141.217 2024-03-26T18:08:11.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:11 smithi115 ceph-mon[36174]: pgmap v185: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-03-26T18:08:11.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:11 smithi049 ceph-mon[32331]: pgmap v185: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-03-26T18:08:12.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:11 smithi171 ceph-mon[36033]: pgmap v185: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-03-26T18:08:13.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:13 smithi115 ceph-mon[36174]: pgmap v186: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 909 op/s 2024-03-26T18:08:13.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:13 smithi049 ceph-mon[32331]: pgmap v186: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 909 op/s 2024-03-26T18:08:14.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:13 smithi171 ceph-mon[36033]: pgmap v186: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 909 op/s 2024-03-26T18:08:15.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 350.0 s, 810.6 tps, lat 123.048 ms stddev 167.603 2024-03-26T18:08:15.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:15 smithi115 ceph-mon[36174]: pgmap v187: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2024-03-26T18:08:15.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:15 smithi049 ceph-mon[32331]: pgmap v187: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2024-03-26T18:08:16.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:15 smithi171 ceph-mon[36033]: pgmap v187: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2024-03-26T18:08:17.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:17 smithi115 ceph-mon[36174]: pgmap v188: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 9.1 MiB/s wr, 457 op/s 2024-03-26T18:08:17.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:17 smithi049 ceph-mon[32331]: pgmap v188: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 9.1 MiB/s wr, 457 op/s 2024-03-26T18:08:18.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:17 smithi171 ceph-mon[36033]: pgmap v188: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 9.1 MiB/s wr, 457 op/s 2024-03-26T18:08:19.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:19 smithi115 ceph-mon[36174]: pgmap v189: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 10 MiB/s wr, 529 op/s 2024-03-26T18:08:19.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:19 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:08:19.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:19 smithi049 ceph-mon[32331]: pgmap v189: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 10 MiB/s wr, 529 op/s 2024-03-26T18:08:19.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:19 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:08:20.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:19 smithi171 ceph-mon[36033]: pgmap v189: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 10 MiB/s wr, 529 op/s 2024-03-26T18:08:20.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:19 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:08:20.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 355.0 s, 645.6 tps, lat 153.491 ms stddev 192.522 2024-03-26T18:08:21.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:21 smithi115 ceph-mon[36174]: pgmap v190: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 12 MiB/s wr, 622 op/s 2024-03-26T18:08:21.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:21 smithi049 ceph-mon[32331]: pgmap v190: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 12 MiB/s wr, 622 op/s 2024-03-26T18:08:22.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:21 smithi171 ceph-mon[36033]: pgmap v190: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 12 MiB/s wr, 622 op/s 2024-03-26T18:08:22.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:22 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:08:22.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:22 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:08:22.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:22 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:08:22.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:22 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:08:22.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:22 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:08:22.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:22 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:08:23.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:22 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:08:23.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:22 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:08:23.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:22 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:08:23.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:23 smithi115 ceph-mon[36174]: pgmap v191: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 9.0 MiB/s wr, 467 op/s 2024-03-26T18:08:23.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:23 smithi049 ceph-mon[32331]: pgmap v191: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 9.0 MiB/s wr, 467 op/s 2024-03-26T18:08:24.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:23 smithi171 ceph-mon[36033]: pgmap v191: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 9.0 MiB/s wr, 467 op/s 2024-03-26T18:08:25.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 360.0 s, 716.2 tps, lat 138.533 ms stddev 180.179 2024-03-26T18:08:25.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:25 smithi115 ceph-mon[36174]: pgmap v192: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 12 MiB/s wr, 641 op/s 2024-03-26T18:08:25.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:25 smithi049 ceph-mon[32331]: pgmap v192: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 12 MiB/s wr, 641 op/s 2024-03-26T18:08:26.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:25 smithi171 ceph-mon[36033]: pgmap v192: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 12 MiB/s wr, 641 op/s 2024-03-26T18:08:27.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:27 smithi115 ceph-mon[36174]: pgmap v193: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.8 MiB/s wr, 461 op/s 2024-03-26T18:08:27.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:27 smithi049 ceph-mon[32331]: pgmap v193: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.8 MiB/s wr, 461 op/s 2024-03-26T18:08:28.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:27 smithi171 ceph-mon[36033]: pgmap v193: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.8 MiB/s wr, 461 op/s 2024-03-26T18:08:29.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:29 smithi115 ceph-mon[36174]: pgmap v194: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.1 MiB/s wr, 547 op/s 2024-03-26T18:08:29.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:29 smithi049 ceph-mon[32331]: pgmap v194: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.1 MiB/s wr, 547 op/s 2024-03-26T18:08:30.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:29 smithi171 ceph-mon[36033]: pgmap v194: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.1 MiB/s wr, 547 op/s 2024-03-26T18:08:30.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 365.0 s, 579.8 tps, lat 171.705 ms stddev 215.477 2024-03-26T18:08:31.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:31 smithi115 ceph-mon[36174]: pgmap v195: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 694 op/s 2024-03-26T18:08:31.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:31 smithi049 ceph-mon[32331]: pgmap v195: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 694 op/s 2024-03-26T18:08:32.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:31 smithi171 ceph-mon[36033]: pgmap v195: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 694 op/s 2024-03-26T18:08:32.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:32 smithi049 ceph-mon[32331]: pgmap v196: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.2 MiB/s wr, 530 op/s 2024-03-26T18:08:33.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:32 smithi171 ceph-mon[36033]: pgmap v196: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.2 MiB/s wr, 530 op/s 2024-03-26T18:08:33.233 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:32 smithi115 ceph-mon[36174]: pgmap v196: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.2 MiB/s wr, 530 op/s 2024-03-26T18:08:35.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 370.0 s, 745.8 tps, lat 136.662 ms stddev 178.909 2024-03-26T18:08:35.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:35 smithi115 ceph-mon[36174]: pgmap v197: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 720 op/s 2024-03-26T18:08:35.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:35 smithi049 ceph-mon[32331]: pgmap v197: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 720 op/s 2024-03-26T18:08:35.602 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:35 smithi171 ceph-mon[36033]: pgmap v197: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 720 op/s 2024-03-26T18:08:37.483 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:37 smithi115 ceph-mon[36174]: pgmap v198: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.7 MiB/s wr, 593 op/s 2024-03-26T18:08:37.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:37 smithi049 ceph-mon[32331]: pgmap v198: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.7 MiB/s wr, 593 op/s 2024-03-26T18:08:37.602 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:37 smithi171 ceph-mon[36033]: pgmap v198: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.7 MiB/s wr, 593 op/s 2024-03-26T18:08:39.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:39 smithi115 ceph-mon[36174]: pgmap v199: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 10 MiB/s wr, 630 op/s 2024-03-26T18:08:39.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:39 smithi049 ceph-mon[32331]: pgmap v199: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 10 MiB/s wr, 630 op/s 2024-03-26T18:08:39.602 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:39 smithi171 ceph-mon[36033]: pgmap v199: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 10 MiB/s wr, 630 op/s 2024-03-26T18:08:40.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 375.0 s, 798.0 tps, lat 126.441 ms stddev 161.851 2024-03-26T18:08:41.483 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:41 smithi115 ceph-mon[36174]: pgmap v200: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 697 op/s 2024-03-26T18:08:41.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:41 smithi049 ceph-mon[32331]: pgmap v200: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 697 op/s 2024-03-26T18:08:41.602 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:41 smithi171 ceph-mon[36033]: pgmap v200: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 697 op/s 2024-03-26T18:08:43.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:43 smithi115 ceph-mon[36174]: pgmap v201: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.5 MiB/s wr, 479 op/s 2024-03-26T18:08:43.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:43 smithi049 ceph-mon[32331]: pgmap v201: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.5 MiB/s wr, 479 op/s 2024-03-26T18:08:43.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:43 smithi171 ceph-mon[36033]: pgmap v201: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.5 MiB/s wr, 479 op/s 2024-03-26T18:08:45.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 380.0 s, 723.8 tps, lat 136.012 ms stddev 148.559 2024-03-26T18:08:45.483 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:45 smithi115 ceph-mon[36174]: pgmap v202: 129 pgs: 129 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-03-26T18:08:45.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:45 smithi049 ceph-mon[32331]: pgmap v202: 129 pgs: 129 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-03-26T18:08:45.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:45 smithi171 ceph-mon[36033]: pgmap v202: 129 pgs: 129 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-03-26T18:08:47.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:47 smithi115 ceph-mon[36174]: pgmap v203: 129 pgs: 129 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.2 MiB/s wr, 522 op/s 2024-03-26T18:08:47.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:47 smithi049 ceph-mon[32331]: pgmap v203: 129 pgs: 129 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.2 MiB/s wr, 522 op/s 2024-03-26T18:08:47.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:47 smithi171 ceph-mon[36033]: pgmap v203: 129 pgs: 129 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.2 MiB/s wr, 522 op/s 2024-03-26T18:08:49.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:49 smithi115 ceph-mon[36174]: pgmap v204: 129 pgs: 129 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.9 MiB/s wr, 541 op/s 2024-03-26T18:08:49.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:49 smithi049 ceph-mon[32331]: pgmap v204: 129 pgs: 129 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.9 MiB/s wr, 541 op/s 2024-03-26T18:08:49.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:49 smithi171 ceph-mon[36033]: pgmap v204: 129 pgs: 129 active+clean; 3.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.9 MiB/s wr, 541 op/s 2024-03-26T18:08:50.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 385.0 s, 661.0 tps, lat 148.818 ms stddev 201.382 2024-03-26T18:08:51.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:51 smithi115 ceph-mon[36174]: pgmap v205: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.7 MiB/s wr, 621 op/s 2024-03-26T18:08:51.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:51 smithi049 ceph-mon[32331]: pgmap v205: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.7 MiB/s wr, 621 op/s 2024-03-26T18:08:51.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:51 smithi171 ceph-mon[36033]: pgmap v205: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.7 MiB/s wr, 621 op/s 2024-03-26T18:08:53.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:53 smithi115 ceph-mon[36174]: pgmap v206: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.8 MiB/s wr, 468 op/s 2024-03-26T18:08:53.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:53 smithi049 ceph-mon[32331]: pgmap v206: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.8 MiB/s wr, 468 op/s 2024-03-26T18:08:53.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:53 smithi171 ceph-mon[36033]: pgmap v206: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.8 MiB/s wr, 468 op/s 2024-03-26T18:08:55.206 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 390.0 s, 509.2 tps, lat 193.072 ms stddev 221.485 2024-03-26T18:08:55.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:55 smithi115 ceph-mon[36174]: pgmap v207: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.4 MiB/s wr, 634 op/s 2024-03-26T18:08:55.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:55 smithi049 ceph-mon[32331]: pgmap v207: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.4 MiB/s wr, 634 op/s 2024-03-26T18:08:55.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:55 smithi171 ceph-mon[36033]: pgmap v207: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.4 MiB/s wr, 634 op/s 2024-03-26T18:08:57.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:57 smithi115 ceph-mon[36174]: pgmap v208: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.4 MiB/s wr, 508 op/s 2024-03-26T18:08:57.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:57 smithi049 ceph-mon[32331]: pgmap v208: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.4 MiB/s wr, 508 op/s 2024-03-26T18:08:57.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:57 smithi171 ceph-mon[36033]: pgmap v208: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.4 MiB/s wr, 508 op/s 2024-03-26T18:08:59.483 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:08:59 smithi115 ceph-mon[36174]: pgmap v209: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.1 MiB/s wr, 625 op/s 2024-03-26T18:08:59.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:08:59 smithi049 ceph-mon[32331]: pgmap v209: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.1 MiB/s wr, 625 op/s 2024-03-26T18:08:59.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:08:59 smithi171 ceph-mon[36033]: pgmap v209: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.1 MiB/s wr, 625 op/s 2024-03-26T18:09:00.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 395.0 s, 533.8 tps, lat 191.375 ms stddev 261.882 2024-03-26T18:09:01.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:01 smithi115 ceph-mon[36174]: pgmap v210: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 984 op/s 2024-03-26T18:09:01.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:01 smithi049 ceph-mon[32331]: pgmap v210: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 984 op/s 2024-03-26T18:09:01.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:01 smithi171 ceph-mon[36033]: pgmap v210: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 984 op/s 2024-03-26T18:09:03.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:03 smithi115 ceph-mon[36174]: pgmap v211: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 7.9 MiB/s wr, 815 op/s 2024-03-26T18:09:03.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:03 smithi049 ceph-mon[32331]: pgmap v211: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 7.9 MiB/s wr, 815 op/s 2024-03-26T18:09:03.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:03 smithi171 ceph-mon[36033]: pgmap v211: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 7.9 MiB/s wr, 815 op/s 2024-03-26T18:09:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 400.0 s, 565.2 tps, lat 175.785 ms stddev 232.972 2024-03-26T18:09:05.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:05 smithi115 ceph-mon[36174]: pgmap v212: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 9.4 MiB/s wr, 986 op/s 2024-03-26T18:09:05.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:05 smithi049 ceph-mon[32331]: pgmap v212: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 9.4 MiB/s wr, 986 op/s 2024-03-26T18:09:05.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:05 smithi171 ceph-mon[36033]: pgmap v212: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 9.4 MiB/s wr, 986 op/s 2024-03-26T18:09:07.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:07 smithi115 ceph-mon[36174]: pgmap v213: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 8.5 MiB/s wr, 866 op/s 2024-03-26T18:09:07.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:07 smithi049 ceph-mon[32331]: pgmap v213: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 8.5 MiB/s wr, 866 op/s 2024-03-26T18:09:07.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:07 smithi171 ceph-mon[36033]: pgmap v213: 129 pgs: 129 active+clean; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 8.5 MiB/s wr, 866 op/s 2024-03-26T18:09:09.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:09 smithi115 ceph-mon[36174]: pgmap v214: 129 pgs: 129 active+clean; 3.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 8.7 MiB/s wr, 886 op/s 2024-03-26T18:09:09.498 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:09 smithi049 ceph-mon[32331]: pgmap v214: 129 pgs: 129 active+clean; 3.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 8.7 MiB/s wr, 886 op/s 2024-03-26T18:09:09.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:09 smithi171 ceph-mon[36033]: pgmap v214: 129 pgs: 129 active+clean; 3.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 8.7 MiB/s wr, 886 op/s 2024-03-26T18:09:10.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 405.0 s, 580.8 tps, lat 167.080 ms stddev 205.486 2024-03-26T18:09:10.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:10 smithi115 ceph-mon[36174]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T18:09:10.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:10 smithi049 ceph-mon[32331]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T18:09:10.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:10 smithi171 ceph-mon[36033]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T18:09:11.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:11 smithi049 ceph-mon[32331]: pgmap v216: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 529 op/s 2024-03-26T18:09:11.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:11 smithi171 ceph-mon[36033]: pgmap v216: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 529 op/s 2024-03-26T18:09:11.732 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:11 smithi115 ceph-mon[36174]: pgmap v216: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 529 op/s 2024-03-26T18:09:13.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:13 smithi049 ceph-mon[32331]: pgmap v217: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 529 op/s 2024-03-26T18:09:13.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:13 smithi171 ceph-mon[36033]: pgmap v217: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 529 op/s 2024-03-26T18:09:13.732 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:13 smithi115 ceph-mon[36174]: pgmap v217: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 3.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 529 op/s 2024-03-26T18:09:15.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 410.0 s, 677.0 tps, lat 155.368 ms stddev 184.872 2024-03-26T18:09:15.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:15 smithi049 ceph-mon[32331]: pgmap v218: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 526 op/s 2024-03-26T18:09:15.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:15 smithi171 ceph-mon[36033]: pgmap v218: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 526 op/s 2024-03-26T18:09:15.732 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:15 smithi115 ceph-mon[36174]: pgmap v218: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 526 op/s 2024-03-26T18:09:17.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:17 smithi171 ceph-mon[36033]: pgmap v219: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.4 MiB/s wr, 521 op/s 2024-03-26T18:09:17.732 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:17 smithi115 ceph-mon[36174]: pgmap v219: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.4 MiB/s wr, 521 op/s 2024-03-26T18:09:17.747 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:17 smithi049 ceph-mon[32331]: pgmap v219: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.4 MiB/s wr, 521 op/s 2024-03-26T18:09:18.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:18 smithi171 ceph-mon[36033]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T18:09:18.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:18 smithi115 ceph-mon[36174]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T18:09:18.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:18 smithi049 ceph-mon[32331]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T18:09:19.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:19 smithi171 ceph-mon[36033]: pgmap v221: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.7 MiB/s wr, 442 op/s 2024-03-26T18:09:19.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:19 smithi115 ceph-mon[36174]: pgmap v221: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.7 MiB/s wr, 442 op/s 2024-03-26T18:09:19.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:19 smithi049 ceph-mon[32331]: pgmap v221: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.7 MiB/s wr, 442 op/s 2024-03-26T18:09:20.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 415.0 s, 696.4 tps, lat 143.150 ms stddev 162.249 2024-03-26T18:09:21.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:21 smithi171 ceph-mon[36033]: pgmap v222: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2024-03-26T18:09:21.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:21 smithi115 ceph-mon[36174]: pgmap v222: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2024-03-26T18:09:21.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:21 smithi049 ceph-mon[32331]: pgmap v222: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2024-03-26T18:09:22.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:22 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:09:22.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:22 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:09:23.017 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:22 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:09:23.874 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:23 smithi171 ceph-mon[36033]: pgmap v223: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2024-03-26T18:09:23.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:23 smithi115 ceph-mon[36174]: pgmap v223: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2024-03-26T18:09:23.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:23 smithi049 ceph-mon[32331]: pgmap v223: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2024-03-26T18:09:24.941 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:24 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:09:24.941 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:24 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:09:24.941 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:24 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:09:24.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:24 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:09:24.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:24 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:09:24.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:24 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:09:25.043 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:24 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:09:25.043 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:24 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:09:25.043 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:24 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:09:25.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 420.0 s, 601.2 tps, lat 167.729 ms stddev 193.439 2024-03-26T18:09:25.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:25 smithi171 ceph-mon[36033]: pgmap v224: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.8 MiB/s wr, 536 op/s 2024-03-26T18:09:25.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:25 smithi049 ceph-mon[32331]: pgmap v224: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.8 MiB/s wr, 536 op/s 2024-03-26T18:09:26.117 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:25 smithi115 ceph-mon[36174]: pgmap v224: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.8 MiB/s wr, 536 op/s 2024-03-26T18:09:27.854 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:27 smithi171 ceph-mon[36033]: pgmap v225: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-03-26T18:09:27.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:27 smithi115 ceph-mon[36174]: pgmap v225: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-03-26T18:09:27.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:27 smithi049 ceph-mon[32331]: pgmap v225: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-03-26T18:09:29.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:29 smithi171 ceph-mon[36033]: pgmap v226: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.9 MiB/s wr, 563 op/s 2024-03-26T18:09:29.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:29 smithi115 ceph-mon[36174]: pgmap v226: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.9 MiB/s wr, 563 op/s 2024-03-26T18:09:29.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:29 smithi049 ceph-mon[32331]: pgmap v226: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.9 MiB/s wr, 563 op/s 2024-03-26T18:09:30.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 425.0 s, 547.6 tps, lat 178.064 ms stddev 218.993 2024-03-26T18:09:31.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:31 smithi115 ceph-mon[36174]: pgmap v227: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.4 MiB/s wr, 624 op/s 2024-03-26T18:09:31.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:31 smithi049 ceph-mon[32331]: pgmap v227: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.4 MiB/s wr, 624 op/s 2024-03-26T18:09:32.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:31 smithi171 ceph-mon[36033]: pgmap v227: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.4 MiB/s wr, 624 op/s 2024-03-26T18:09:33.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:33 smithi115 ceph-mon[36174]: pgmap v228: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.8 MiB/s wr, 468 op/s 2024-03-26T18:09:33.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:33 smithi049 ceph-mon[32331]: pgmap v228: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.8 MiB/s wr, 468 op/s 2024-03-26T18:09:34.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:33 smithi171 ceph-mon[36033]: pgmap v228: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.8 MiB/s wr, 468 op/s 2024-03-26T18:09:35.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 430.0 s, 584.4 tps, lat 173.741 ms stddev 268.662 2024-03-26T18:09:35.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:35 smithi115 ceph-mon[36174]: pgmap v229: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.9 MiB/s wr, 662 op/s 2024-03-26T18:09:35.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:35 smithi049 ceph-mon[32331]: pgmap v229: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.9 MiB/s wr, 662 op/s 2024-03-26T18:09:36.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:35 smithi171 ceph-mon[36033]: pgmap v229: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.9 MiB/s wr, 662 op/s 2024-03-26T18:09:37.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:37 smithi115 ceph-mon[36174]: pgmap v230: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 539 op/s 2024-03-26T18:09:37.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:37 smithi049 ceph-mon[32331]: pgmap v230: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 539 op/s 2024-03-26T18:09:38.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:37 smithi171 ceph-mon[36033]: pgmap v230: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 539 op/s 2024-03-26T18:09:39.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:39 smithi115 ceph-mon[36174]: pgmap v231: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-03-26T18:09:39.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:39 smithi049 ceph-mon[32331]: pgmap v231: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-03-26T18:09:40.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:39 smithi171 ceph-mon[36033]: pgmap v231: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-03-26T18:09:40.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 435.0 s, 567.0 tps, lat 173.750 ms stddev 222.975 2024-03-26T18:09:41.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:41 smithi115 ceph-mon[36174]: pgmap v232: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.6 MiB/s wr, 672 op/s 2024-03-26T18:09:41.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:41 smithi049 ceph-mon[32331]: pgmap v232: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.6 MiB/s wr, 672 op/s 2024-03-26T18:09:42.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:41 smithi171 ceph-mon[36033]: pgmap v232: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.6 MiB/s wr, 672 op/s 2024-03-26T18:09:43.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:43 smithi115 ceph-mon[36174]: pgmap v233: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.1 MiB/s wr, 494 op/s 2024-03-26T18:09:43.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:43 smithi049 ceph-mon[32331]: pgmap v233: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.1 MiB/s wr, 494 op/s 2024-03-26T18:09:44.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:43 smithi171 ceph-mon[36033]: pgmap v233: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.1 MiB/s wr, 494 op/s 2024-03-26T18:09:45.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 440.0 s, 553.6 tps, lat 184.509 ms stddev 239.571 2024-03-26T18:09:45.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:45 smithi115 ceph-mon[36174]: pgmap v234: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.3 MiB/s wr, 752 op/s 2024-03-26T18:09:45.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:45 smithi049 ceph-mon[32331]: pgmap v234: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.3 MiB/s wr, 752 op/s 2024-03-26T18:09:46.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:45 smithi171 ceph-mon[36033]: pgmap v234: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.3 MiB/s wr, 752 op/s 2024-03-26T18:09:47.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:47 smithi115 ceph-mon[36174]: pgmap v235: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 637 op/s 2024-03-26T18:09:47.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:47 smithi049 ceph-mon[32331]: pgmap v235: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 637 op/s 2024-03-26T18:09:48.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:47 smithi171 ceph-mon[36033]: pgmap v235: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 637 op/s 2024-03-26T18:09:49.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:49 smithi115 ceph-mon[36174]: pgmap v236: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 631 op/s 2024-03-26T18:09:49.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:49 smithi049 ceph-mon[32331]: pgmap v236: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 631 op/s 2024-03-26T18:09:50.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:49 smithi171 ceph-mon[36033]: pgmap v236: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 631 op/s 2024-03-26T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 445.0 s, 506.0 tps, lat 196.724 ms stddev 226.304 2024-03-26T18:09:51.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:51 smithi115 ceph-mon[36174]: pgmap v237: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-03-26T18:09:51.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:51 smithi049 ceph-mon[32331]: pgmap v237: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-03-26T18:09:52.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:51 smithi171 ceph-mon[36033]: pgmap v237: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-03-26T18:09:53.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:53 smithi115 ceph-mon[36174]: pgmap v238: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 546 op/s 2024-03-26T18:09:53.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:53 smithi049 ceph-mon[32331]: pgmap v238: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 546 op/s 2024-03-26T18:09:54.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:53 smithi171 ceph-mon[36033]: pgmap v238: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 546 op/s 2024-03-26T18:09:55.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 450.0 s, 626.0 tps, lat 157.177 ms stddev 171.537 2024-03-26T18:09:55.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:55 smithi115 ceph-mon[36174]: pgmap v239: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 705 op/s 2024-03-26T18:09:55.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:55 smithi049 ceph-mon[32331]: pgmap v239: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 705 op/s 2024-03-26T18:09:56.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:55 smithi171 ceph-mon[36033]: pgmap v239: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 705 op/s 2024-03-26T18:09:57.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:57 smithi115 ceph-mon[36174]: pgmap v240: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 501 op/s 2024-03-26T18:09:57.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:57 smithi049 ceph-mon[32331]: pgmap v240: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 501 op/s 2024-03-26T18:09:58.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:57 smithi171 ceph-mon[36033]: pgmap v240: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 501 op/s 2024-03-26T18:09:59.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:09:59 smithi115 ceph-mon[36174]: pgmap v241: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-03-26T18:09:59.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:09:59 smithi049 ceph-mon[32331]: pgmap v241: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-03-26T18:10:00.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:09:59 smithi171 ceph-mon[36033]: pgmap v241: 129 pgs: 129 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-03-26T18:10:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 455.0 s, 571.4 tps, lat 173.875 ms stddev 191.715 2024-03-26T18:10:00.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:00 smithi115 ceph-mon[36174]: overall HEALTH_OK 2024-03-26T18:10:00.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:00 smithi049 ceph-mon[32331]: overall HEALTH_OK 2024-03-26T18:10:01.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:00 smithi171 ceph-mon[36033]: overall HEALTH_OK 2024-03-26T18:10:01.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:01 smithi115 ceph-mon[36174]: pgmap v242: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-03-26T18:10:01.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:01 smithi049 ceph-mon[32331]: pgmap v242: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-03-26T18:10:02.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:01 smithi171 ceph-mon[36033]: pgmap v242: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-03-26T18:10:03.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:03 smithi115 ceph-mon[36174]: pgmap v243: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 452 op/s 2024-03-26T18:10:03.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:03 smithi049 ceph-mon[32331]: pgmap v243: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 452 op/s 2024-03-26T18:10:04.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:03 smithi171 ceph-mon[36033]: pgmap v243: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 452 op/s 2024-03-26T18:10:05.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 460.0 s, 474.2 tps, lat 211.960 ms stddev 260.968 2024-03-26T18:10:05.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:05 smithi115 ceph-mon[36174]: pgmap v244: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.9 MiB/s wr, 644 op/s 2024-03-26T18:10:05.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:05 smithi049 ceph-mon[32331]: pgmap v244: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.9 MiB/s wr, 644 op/s 2024-03-26T18:10:06.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:05 smithi171 ceph-mon[36033]: pgmap v244: 129 pgs: 129 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.9 MiB/s wr, 644 op/s 2024-03-26T18:10:07.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:07 smithi115 ceph-mon[36174]: pgmap v245: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.0 MiB/s wr, 546 op/s 2024-03-26T18:10:07.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:07 smithi049 ceph-mon[32331]: pgmap v245: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.0 MiB/s wr, 546 op/s 2024-03-26T18:10:08.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:07 smithi171 ceph-mon[36033]: pgmap v245: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.0 MiB/s wr, 546 op/s 2024-03-26T18:10:09.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:09 smithi115 ceph-mon[36174]: pgmap v246: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 532 op/s 2024-03-26T18:10:09.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:09 smithi115 ceph-mon[36174]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T18:10:09.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:09 smithi049 ceph-mon[32331]: pgmap v246: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 532 op/s 2024-03-26T18:10:09.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:09 smithi049 ceph-mon[32331]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T18:10:10.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:09 smithi171 ceph-mon[36033]: pgmap v246: 129 pgs: 129 active+clean; 3.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 532 op/s 2024-03-26T18:10:10.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:09 smithi171 ceph-mon[36033]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T18:10:10.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 465.0 s, 458.8 tps, lat 218.667 ms stddev 267.344 2024-03-26T18:10:11.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:11 smithi115 ceph-mon[36174]: pgmap v248: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 3.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.2 MiB/s wr, 580 op/s 2024-03-26T18:10:11.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:11 smithi049 ceph-mon[32331]: pgmap v248: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 3.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.2 MiB/s wr, 580 op/s 2024-03-26T18:10:12.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:11 smithi171 ceph-mon[36033]: pgmap v248: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 3.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.2 MiB/s wr, 580 op/s 2024-03-26T18:10:12.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:12 smithi115 ceph-mon[36174]: pgmap v249: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.8 MiB/s wr, 551 op/s 2024-03-26T18:10:12.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:12 smithi049 ceph-mon[32331]: pgmap v249: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.8 MiB/s wr, 551 op/s 2024-03-26T18:10:13.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:12 smithi171 ceph-mon[36033]: pgmap v249: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 3.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.8 MiB/s wr, 551 op/s 2024-03-26T18:10:15.207 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 470.0 s, 636.0 tps, lat 162.158 ms stddev 187.574 2024-03-26T18:10:15.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:15 smithi115 ceph-mon[36174]: pgmap v250: 129 pgs: 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 126 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 498 op/s 2024-03-26T18:10:15.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:15 smithi049 ceph-mon[32331]: pgmap v250: 129 pgs: 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 126 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 498 op/s 2024-03-26T18:10:15.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:15 smithi171 ceph-mon[36033]: pgmap v250: 129 pgs: 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 126 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 498 op/s 2024-03-26T18:10:17.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:17 smithi115 ceph-mon[36174]: pgmap v251: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.6 MiB/s wr, 495 op/s 2024-03-26T18:10:17.498 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:17 smithi049 ceph-mon[32331]: pgmap v251: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.6 MiB/s wr, 495 op/s 2024-03-26T18:10:17.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:17 smithi171 ceph-mon[36033]: pgmap v251: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.6 MiB/s wr, 495 op/s 2024-03-26T18:10:18.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:18 smithi171 ceph-mon[36033]: osdmap e99: 12 total, 12 up, 12 in 2024-03-26T18:10:18.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:18 smithi115 ceph-mon[36174]: osdmap e99: 12 total, 12 up, 12 in 2024-03-26T18:10:18.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:18 smithi049 ceph-mon[32331]: osdmap e99: 12 total, 12 up, 12 in 2024-03-26T18:10:19.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:19 smithi171 ceph-mon[36033]: pgmap v253: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-03-26T18:10:19.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:19 smithi115 ceph-mon[36174]: pgmap v253: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-03-26T18:10:19.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:19 smithi049 ceph-mon[32331]: pgmap v253: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-03-26T18:10:20.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 475.0 s, 664.2 tps, lat 146.696 ms stddev 177.377 2024-03-26T18:10:21.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:21 smithi171 ceph-mon[36033]: pgmap v254: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-03-26T18:10:21.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:21 smithi115 ceph-mon[36174]: pgmap v254: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-03-26T18:10:21.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:21 smithi049 ceph-mon[32331]: pgmap v254: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-03-26T18:10:23.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:23 smithi171 ceph-mon[36033]: pgmap v255: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 521 op/s 2024-03-26T18:10:23.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:23 smithi115 ceph-mon[36174]: pgmap v255: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 521 op/s 2024-03-26T18:10:23.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:23 smithi049 ceph-mon[32331]: pgmap v255: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 521 op/s 2024-03-26T18:10:25.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 480.0 s, 543.8 tps, lat 188.272 ms stddev 217.699 2024-03-26T18:10:25.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:25 smithi171 ceph-mon[36033]: pgmap v256: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.6 MiB/s wr, 510 op/s 2024-03-26T18:10:25.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:25 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:10:25.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:25 smithi115 ceph-mon[36174]: pgmap v256: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.6 MiB/s wr, 510 op/s 2024-03-26T18:10:25.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:25 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:10:25.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:25 smithi049 ceph-mon[32331]: pgmap v256: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.6 MiB/s wr, 510 op/s 2024-03-26T18:10:25.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:25 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:10:27.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:27 smithi115 ceph-mon[36174]: pgmap v257: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.9 MiB/s wr, 532 op/s 2024-03-26T18:10:27.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:27 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:10:27.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:27 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:10:27.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:27 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:10:27.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:27 smithi049 ceph-mon[32331]: pgmap v257: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.9 MiB/s wr, 532 op/s 2024-03-26T18:10:27.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:27 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:10:27.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:27 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:10:27.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:27 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:10:28.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:27 smithi171 ceph-mon[36033]: pgmap v257: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.9 MiB/s wr, 532 op/s 2024-03-26T18:10:28.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:27 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:10:28.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:27 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:10:28.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:27 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:10:29.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:29 smithi115 ceph-mon[36174]: pgmap v258: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.8 MiB/s wr, 519 op/s 2024-03-26T18:10:29.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:29 smithi049 ceph-mon[32331]: pgmap v258: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.8 MiB/s wr, 519 op/s 2024-03-26T18:10:30.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:29 smithi171 ceph-mon[36033]: pgmap v258: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.8 MiB/s wr, 519 op/s 2024-03-26T18:10:30.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 485.0 s, 495.4 tps, lat 195.341 ms stddev 214.255 2024-03-26T18:10:31.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:31 smithi115 ceph-mon[36174]: pgmap v259: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-03-26T18:10:31.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:31 smithi049 ceph-mon[32331]: pgmap v259: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-03-26T18:10:32.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:31 smithi171 ceph-mon[36033]: pgmap v259: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-03-26T18:10:33.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:33 smithi115 ceph-mon[36174]: pgmap v260: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.7 MiB/s wr, 490 op/s 2024-03-26T18:10:33.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:33 smithi049 ceph-mon[32331]: pgmap v260: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.7 MiB/s wr, 490 op/s 2024-03-26T18:10:34.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:33 smithi171 ceph-mon[36033]: pgmap v260: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.7 MiB/s wr, 490 op/s 2024-03-26T18:10:35.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 490.0 s, 613.0 tps, lat 163.120 ms stddev 181.134 2024-03-26T18:10:35.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:35 smithi115 ceph-mon[36174]: pgmap v261: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-03-26T18:10:35.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:35 smithi049 ceph-mon[32331]: pgmap v261: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-03-26T18:10:36.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:35 smithi171 ceph-mon[36033]: pgmap v261: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-03-26T18:10:37.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:37 smithi115 ceph-mon[36174]: pgmap v262: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.9 MiB/s wr, 542 op/s 2024-03-26T18:10:37.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:37 smithi049 ceph-mon[32331]: pgmap v262: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.9 MiB/s wr, 542 op/s 2024-03-26T18:10:38.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:37 smithi171 ceph-mon[36033]: pgmap v262: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.9 MiB/s wr, 542 op/s 2024-03-26T18:10:39.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:39 smithi115 ceph-mon[36174]: pgmap v263: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-03-26T18:10:39.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:39 smithi049 ceph-mon[32331]: pgmap v263: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-03-26T18:10:40.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:39 smithi171 ceph-mon[36033]: pgmap v263: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-03-26T18:10:40.205 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 495.0 s, 495.3 tps, lat 201.360 ms stddev 220.772 2024-03-26T18:10:41.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:41 smithi115 ceph-mon[36174]: pgmap v264: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 609 op/s 2024-03-26T18:10:41.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:41 smithi049 ceph-mon[32331]: pgmap v264: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 609 op/s 2024-03-26T18:10:42.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:41 smithi171 ceph-mon[36033]: pgmap v264: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 609 op/s 2024-03-26T18:10:43.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:43 smithi115 ceph-mon[36174]: pgmap v265: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 527 op/s 2024-03-26T18:10:43.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:43 smithi049 ceph-mon[32331]: pgmap v265: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 527 op/s 2024-03-26T18:10:44.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:43 smithi171 ceph-mon[36033]: pgmap v265: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 527 op/s 2024-03-26T18:10:45.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 500.0 s, 476.7 tps, lat 211.787 ms stddev 236.764 2024-03-26T18:10:45.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:45 smithi115 ceph-mon[36174]: pgmap v266: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 602 op/s 2024-03-26T18:10:45.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:45 smithi049 ceph-mon[32331]: pgmap v266: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 602 op/s 2024-03-26T18:10:46.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:45 smithi171 ceph-mon[36033]: pgmap v266: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 602 op/s 2024-03-26T18:10:47.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:47 smithi115 ceph-mon[36174]: pgmap v267: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 510 op/s 2024-03-26T18:10:47.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:47 smithi049 ceph-mon[32331]: pgmap v267: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 510 op/s 2024-03-26T18:10:48.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:47 smithi171 ceph-mon[36033]: pgmap v267: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 510 op/s 2024-03-26T18:10:49.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:49 smithi115 ceph-mon[36174]: pgmap v268: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.1 MiB/s wr, 463 op/s 2024-03-26T18:10:49.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:49 smithi049 ceph-mon[32331]: pgmap v268: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.1 MiB/s wr, 463 op/s 2024-03-26T18:10:50.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:49 smithi171 ceph-mon[36033]: pgmap v268: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.1 MiB/s wr, 463 op/s 2024-03-26T18:10:50.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 505.0 s, 430.8 tps, lat 216.915 ms stddev 258.693 2024-03-26T18:10:51.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:51 smithi115 ceph-mon[36174]: pgmap v269: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 588 op/s 2024-03-26T18:10:51.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:51 smithi049 ceph-mon[32331]: pgmap v269: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 588 op/s 2024-03-26T18:10:52.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:51 smithi171 ceph-mon[36033]: pgmap v269: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 588 op/s 2024-03-26T18:10:53.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:53 smithi115 ceph-mon[36174]: pgmap v270: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-03-26T18:10:53.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:53 smithi049 ceph-mon[32331]: pgmap v270: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-03-26T18:10:54.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:53 smithi171 ceph-mon[36033]: pgmap v270: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-03-26T18:10:55.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 510.0 s, 629.0 tps, lat 171.171 ms stddev 235.144 2024-03-26T18:10:55.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:55 smithi115 ceph-mon[36174]: pgmap v271: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 MiB/s wr, 560 op/s 2024-03-26T18:10:55.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:55 smithi049 ceph-mon[32331]: pgmap v271: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 MiB/s wr, 560 op/s 2024-03-26T18:10:56.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:55 smithi171 ceph-mon[36033]: pgmap v271: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 MiB/s wr, 560 op/s 2024-03-26T18:10:57.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:57 smithi115 ceph-mon[36174]: pgmap v272: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.6 MiB/s wr, 525 op/s 2024-03-26T18:10:57.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:57 smithi049 ceph-mon[32331]: pgmap v272: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.6 MiB/s wr, 525 op/s 2024-03-26T18:10:58.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:57 smithi171 ceph-mon[36033]: pgmap v272: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.6 MiB/s wr, 525 op/s 2024-03-26T18:10:59.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:10:59 smithi115 ceph-mon[36174]: pgmap v273: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 484 op/s 2024-03-26T18:10:59.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:10:59 smithi049 ceph-mon[32331]: pgmap v273: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 484 op/s 2024-03-26T18:11:00.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:10:59 smithi171 ceph-mon[36033]: pgmap v273: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 484 op/s 2024-03-26T18:11:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 515.0 s, 497.4 tps, lat 199.285 ms stddev 226.957 2024-03-26T18:11:01.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:01 smithi115 ceph-mon[36174]: pgmap v274: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 594 op/s 2024-03-26T18:11:01.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:01 smithi049 ceph-mon[32331]: pgmap v274: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 594 op/s 2024-03-26T18:11:02.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:01 smithi171 ceph-mon[36033]: pgmap v274: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 594 op/s 2024-03-26T18:11:03.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:03 smithi049 ceph-mon[32331]: pgmap v275: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-03-26T18:11:04.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:03 smithi171 ceph-mon[36033]: pgmap v275: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-03-26T18:11:04.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:03 smithi115 ceph-mon[36174]: pgmap v275: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-03-26T18:11:05.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:04 smithi171 ceph-mon[36033]: pgmap v276: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 532 op/s 2024-03-26T18:11:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 520.0 s, 487.0 tps, lat 208.409 ms stddev 254.503 2024-03-26T18:11:05.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:04 smithi115 ceph-mon[36174]: pgmap v276: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 532 op/s 2024-03-26T18:11:05.246 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:04 smithi049 ceph-mon[32331]: pgmap v276: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 532 op/s 2024-03-26T18:11:07.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:07 smithi115 ceph-mon[36174]: pgmap v277: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.4 MiB/s wr, 518 op/s 2024-03-26T18:11:07.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:07 smithi049 ceph-mon[32331]: pgmap v277: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.4 MiB/s wr, 518 op/s 2024-03-26T18:11:07.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:07 smithi171 ceph-mon[36033]: pgmap v277: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.4 MiB/s wr, 518 op/s 2024-03-26T18:11:09.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:09 smithi115 ceph-mon[36174]: pgmap v278: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.2 MiB/s wr, 467 op/s 2024-03-26T18:11:09.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:09 smithi049 ceph-mon[32331]: pgmap v278: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.2 MiB/s wr, 467 op/s 2024-03-26T18:11:09.601 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:09 smithi171 ceph-mon[36033]: pgmap v278: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.2 MiB/s wr, 467 op/s 2024-03-26T18:11:10.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 525.0 s, 446.6 tps, lat 193.807 ms stddev 223.012 2024-03-26T18:11:10.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:10 smithi115 ceph-mon[36174]: osdmap e100: 12 total, 12 up, 12 in 2024-03-26T18:11:10.498 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:10 smithi049 ceph-mon[32331]: osdmap e100: 12 total, 12 up, 12 in 2024-03-26T18:11:10.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:10 smithi171 ceph-mon[36033]: osdmap e100: 12 total, 12 up, 12 in 2024-03-26T18:11:11.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:11 smithi115 ceph-mon[36174]: pgmap v280: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-03-26T18:11:11.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:11 smithi049 ceph-mon[32331]: pgmap v280: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-03-26T18:11:11.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:11 smithi171 ceph-mon[36033]: pgmap v280: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-03-26T18:11:13.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:13 smithi115 ceph-mon[36174]: pgmap v281: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 500 op/s 2024-03-26T18:11:13.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:13 smithi049 ceph-mon[32331]: pgmap v281: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 500 op/s 2024-03-26T18:11:13.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:13 smithi171 ceph-mon[36033]: pgmap v281: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 500 op/s 2024-03-26T18:11:15.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 530.0 s, 600.4 tps, lat 190.700 ms stddev 246.535 2024-03-26T18:11:15.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:15 smithi115 ceph-mon[36174]: pgmap v282: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.5 MiB/s wr, 514 op/s 2024-03-26T18:11:15.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:15 smithi049 ceph-mon[32331]: pgmap v282: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.5 MiB/s wr, 514 op/s 2024-03-26T18:11:15.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:15 smithi171 ceph-mon[36033]: pgmap v282: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.5 MiB/s wr, 514 op/s 2024-03-26T18:11:17.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:17 smithi115 ceph-mon[36174]: pgmap v283: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-03-26T18:11:17.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:17 smithi049 ceph-mon[32331]: pgmap v283: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-03-26T18:11:17.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:17 smithi171 ceph-mon[36033]: pgmap v283: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.6 MiB/s wr, 514 op/s 2024-03-26T18:11:18.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:18 smithi171 ceph-mon[36033]: osdmap e101: 12 total, 12 up, 12 in 2024-03-26T18:11:18.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:18 smithi115 ceph-mon[36174]: osdmap e101: 12 total, 12 up, 12 in 2024-03-26T18:11:18.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:18 smithi049 ceph-mon[32331]: osdmap e101: 12 total, 12 up, 12 in 2024-03-26T18:11:19.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:19 smithi171 ceph-mon[36033]: pgmap v285: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.8 MiB/s wr, 447 op/s 2024-03-26T18:11:19.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:19 smithi115 ceph-mon[36174]: pgmap v285: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.8 MiB/s wr, 447 op/s 2024-03-26T18:11:19.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:19 smithi049 ceph-mon[32331]: pgmap v285: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.8 MiB/s wr, 447 op/s 2024-03-26T18:11:20.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 535.0 s, 500.2 tps, lat 191.238 ms stddev 204.100 2024-03-26T18:11:21.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:21 smithi115 ceph-mon[36174]: pgmap v286: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 505 op/s 2024-03-26T18:11:21.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:21 smithi049 ceph-mon[32331]: pgmap v286: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 505 op/s 2024-03-26T18:11:22.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:21 smithi171 ceph-mon[36033]: pgmap v286: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 505 op/s 2024-03-26T18:11:23.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:23 smithi115 ceph-mon[36174]: pgmap v287: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2024-03-26T18:11:23.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:23 smithi049 ceph-mon[32331]: pgmap v287: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2024-03-26T18:11:24.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:23 smithi171 ceph-mon[36033]: pgmap v287: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2024-03-26T18:11:25.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 540.0 s, 480.2 tps, lat 206.254 ms stddev 253.250 2024-03-26T18:11:25.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:25 smithi171 ceph-mon[36033]: pgmap v288: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 526 op/s 2024-03-26T18:11:25.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:25 smithi115 ceph-mon[36174]: pgmap v288: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 526 op/s 2024-03-26T18:11:25.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:25 smithi049 ceph-mon[32331]: pgmap v288: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.7 MiB/s wr, 526 op/s 2024-03-26T18:11:27.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:27 smithi115 ceph-mon[36174]: pgmap v289: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.7 MiB/s wr, 526 op/s 2024-03-26T18:11:27.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:27 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:11:27.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:27 smithi049 ceph-mon[32331]: pgmap v289: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.7 MiB/s wr, 526 op/s 2024-03-26T18:11:27.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:27 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:11:28.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:27 smithi171 ceph-mon[36033]: pgmap v289: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.7 MiB/s wr, 526 op/s 2024-03-26T18:11:28.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:27 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:11:28.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:28 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:11:28.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:28 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:11:28.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:28 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:11:28.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:28 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:11:28.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:28 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:11:28.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:28 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:11:29.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:28 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:11:29.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:28 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:11:29.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:28 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:11:29.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:29 smithi115 ceph-mon[36174]: pgmap v290: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.6 MiB/s wr, 518 op/s 2024-03-26T18:11:29.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:29 smithi049 ceph-mon[32331]: pgmap v290: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.6 MiB/s wr, 518 op/s 2024-03-26T18:11:30.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:29 smithi171 ceph-mon[36033]: pgmap v290: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.6 MiB/s wr, 518 op/s 2024-03-26T18:11:30.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 545.0 s, 520.8 tps, lat 194.131 ms stddev 240.011 2024-03-26T18:11:31.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:31 smithi115 ceph-mon[36174]: pgmap v291: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.0 MiB/s wr, 559 op/s 2024-03-26T18:11:31.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:31 smithi049 ceph-mon[32331]: pgmap v291: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.0 MiB/s wr, 559 op/s 2024-03-26T18:11:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:31 smithi171 ceph-mon[36033]: pgmap v291: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.0 MiB/s wr, 559 op/s 2024-03-26T18:11:33.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:33 smithi115 ceph-mon[36174]: pgmap v292: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.1 MiB/s wr, 471 op/s 2024-03-26T18:11:33.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:33 smithi049 ceph-mon[32331]: pgmap v292: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.1 MiB/s wr, 471 op/s 2024-03-26T18:11:34.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:33 smithi171 ceph-mon[36033]: pgmap v292: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.1 MiB/s wr, 471 op/s 2024-03-26T18:11:35.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 550.0 s, 520.2 tps, lat 198.217 ms stddev 228.835 2024-03-26T18:11:35.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:35 smithi115 ceph-mon[36174]: pgmap v293: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-03-26T18:11:35.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:35 smithi049 ceph-mon[32331]: pgmap v293: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-03-26T18:11:36.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:35 smithi171 ceph-mon[36033]: pgmap v293: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.7 MiB/s wr, 535 op/s 2024-03-26T18:11:37.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:37 smithi115 ceph-mon[36174]: pgmap v294: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.6 MiB/s wr, 517 op/s 2024-03-26T18:11:37.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:37 smithi049 ceph-mon[32331]: pgmap v294: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.6 MiB/s wr, 517 op/s 2024-03-26T18:11:38.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:37 smithi171 ceph-mon[36033]: pgmap v294: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.6 MiB/s wr, 517 op/s 2024-03-26T18:11:39.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:39 smithi115 ceph-mon[36174]: pgmap v295: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.7 MiB/s wr, 435 op/s 2024-03-26T18:11:39.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:39 smithi049 ceph-mon[32331]: pgmap v295: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.7 MiB/s wr, 435 op/s 2024-03-26T18:11:40.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:39 smithi171 ceph-mon[36033]: pgmap v295: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.7 MiB/s wr, 435 op/s 2024-03-26T18:11:40.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 555.0 s, 483.8 tps, lat 204.412 ms stddev 230.754 2024-03-26T18:11:41.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:41 smithi115 ceph-mon[36174]: pgmap v296: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 540 op/s 2024-03-26T18:11:41.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:41 smithi049 ceph-mon[32331]: pgmap v296: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 540 op/s 2024-03-26T18:11:42.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:41 smithi171 ceph-mon[36033]: pgmap v296: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 540 op/s 2024-03-26T18:11:43.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:43 smithi115 ceph-mon[36174]: pgmap v297: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.7 MiB/s wr, 476 op/s 2024-03-26T18:11:43.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:43 smithi049 ceph-mon[32331]: pgmap v297: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.7 MiB/s wr, 476 op/s 2024-03-26T18:11:44.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:43 smithi171 ceph-mon[36033]: pgmap v297: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.7 MiB/s wr, 476 op/s 2024-03-26T18:11:45.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 560.0 s, 479.6 tps, lat 207.558 ms stddev 280.369 2024-03-26T18:11:45.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:45 smithi115 ceph-mon[36174]: pgmap v298: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-03-26T18:11:45.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:45 smithi049 ceph-mon[32331]: pgmap v298: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-03-26T18:11:46.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:45 smithi171 ceph-mon[36033]: pgmap v298: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.1 MiB/s wr, 524 op/s 2024-03-26T18:11:47.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:47 smithi115 ceph-mon[36174]: pgmap v299: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-03-26T18:11:47.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:47 smithi049 ceph-mon[32331]: pgmap v299: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-03-26T18:11:48.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:47 smithi171 ceph-mon[36033]: pgmap v299: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-03-26T18:11:49.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:49 smithi115 ceph-mon[36174]: pgmap v300: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.3 MiB/s wr, 437 op/s 2024-03-26T18:11:49.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:49 smithi049 ceph-mon[32331]: pgmap v300: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.3 MiB/s wr, 437 op/s 2024-03-26T18:11:50.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:49 smithi171 ceph-mon[36033]: pgmap v300: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.3 MiB/s wr, 437 op/s 2024-03-26T18:11:50.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 565.0 s, 500.0 tps, lat 198.111 ms stddev 221.943 2024-03-26T18:11:51.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:51 smithi115 ceph-mon[36174]: pgmap v301: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-03-26T18:11:52.000 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:51 smithi049 ceph-mon[32331]: pgmap v301: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-03-26T18:11:52.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:51 smithi171 ceph-mon[36033]: pgmap v301: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-03-26T18:11:53.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:53 smithi115 ceph-mon[36174]: pgmap v302: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-03-26T18:11:53.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:53 smithi049 ceph-mon[32331]: pgmap v302: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-03-26T18:11:54.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:53 smithi171 ceph-mon[36033]: pgmap v302: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-03-26T18:11:55.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 570.0 s, 578.4 tps, lat 176.671 ms stddev 202.577 2024-03-26T18:11:55.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:55 smithi115 ceph-mon[36174]: pgmap v303: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-03-26T18:11:55.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:55 smithi049 ceph-mon[32331]: pgmap v303: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-03-26T18:11:56.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:55 smithi171 ceph-mon[36033]: pgmap v303: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-03-26T18:11:57.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:57 smithi115 ceph-mon[36174]: pgmap v304: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-03-26T18:11:57.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:57 smithi049 ceph-mon[32331]: pgmap v304: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-03-26T18:11:58.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:57 smithi171 ceph-mon[36033]: pgmap v304: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-03-26T18:11:59.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:11:59 smithi115 ceph-mon[36174]: pgmap v305: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.8 MiB/s wr, 442 op/s 2024-03-26T18:11:59.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:11:59 smithi049 ceph-mon[32331]: pgmap v305: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.8 MiB/s wr, 442 op/s 2024-03-26T18:12:00.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:11:59 smithi171 ceph-mon[36033]: pgmap v305: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.8 MiB/s wr, 442 op/s 2024-03-26T18:12:00.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 575.0 s, 541.0 tps, lat 182.072 ms stddev 205.626 2024-03-26T18:12:01.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:01 smithi115 ceph-mon[36174]: pgmap v306: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-03-26T18:12:01.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:01 smithi049 ceph-mon[32331]: pgmap v306: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-03-26T18:12:02.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:01 smithi171 ceph-mon[36033]: pgmap v306: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-03-26T18:12:03.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:03 smithi115 ceph-mon[36174]: pgmap v307: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 477 op/s 2024-03-26T18:12:03.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:03 smithi049 ceph-mon[32331]: pgmap v307: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 477 op/s 2024-03-26T18:12:04.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:03 smithi171 ceph-mon[36033]: pgmap v307: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 477 op/s 2024-03-26T18:12:05.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 580.0 s, 555.2 tps, lat 179.307 ms stddev 233.299 2024-03-26T18:12:05.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:05 smithi049 ceph-mon[32331]: pgmap v308: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.6 MiB/s wr, 534 op/s 2024-03-26T18:12:06.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:05 smithi171 ceph-mon[36033]: pgmap v308: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.6 MiB/s wr, 534 op/s 2024-03-26T18:12:06.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:05 smithi115 ceph-mon[36174]: pgmap v308: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.6 MiB/s wr, 534 op/s 2024-03-26T18:12:07.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:07 smithi049 ceph-mon[32331]: pgmap v309: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.2 MiB/s wr, 482 op/s 2024-03-26T18:12:08.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:07 smithi171 ceph-mon[36033]: pgmap v309: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.2 MiB/s wr, 482 op/s 2024-03-26T18:12:08.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:07 smithi115 ceph-mon[36174]: pgmap v309: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.2 MiB/s wr, 482 op/s 2024-03-26T18:12:10.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:09 smithi115 ceph-mon[36174]: pgmap v310: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.3 MiB/s wr, 500 op/s 2024-03-26T18:12:10.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:09 smithi115 ceph-mon[36174]: osdmap e102: 12 total, 12 up, 12 in 2024-03-26T18:12:10.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:09 smithi171 ceph-mon[36033]: pgmap v310: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.3 MiB/s wr, 500 op/s 2024-03-26T18:12:10.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:09 smithi171 ceph-mon[36033]: osdmap e102: 12 total, 12 up, 12 in 2024-03-26T18:12:10.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 585.0 s, 600.0 tps, lat 168.272 ms stddev 198.156 2024-03-26T18:12:10.246 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:09 smithi049 ceph-mon[32331]: pgmap v310: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.3 MiB/s wr, 500 op/s 2024-03-26T18:12:10.246 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:09 smithi049 ceph-mon[32331]: osdmap e102: 12 total, 12 up, 12 in 2024-03-26T18:12:12.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:11 smithi171 ceph-mon[36033]: pgmap v312: 129 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 105 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 510 op/s 2024-03-26T18:12:12.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:11 smithi115 ceph-mon[36174]: pgmap v312: 129 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 105 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 510 op/s 2024-03-26T18:12:12.246 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:11 smithi049 ceph-mon[32331]: pgmap v312: 129 pgs: 7 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 105 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 510 op/s 2024-03-26T18:12:13.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:12 smithi171 ceph-mon[36033]: pgmap v313: 129 pgs: 9 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 100 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.7 MiB/s wr, 530 op/s 2024-03-26T18:12:13.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:12 smithi115 ceph-mon[36174]: pgmap v313: 129 pgs: 9 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 100 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.7 MiB/s wr, 530 op/s 2024-03-26T18:12:13.246 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:12 smithi049 ceph-mon[32331]: pgmap v313: 129 pgs: 9 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 100 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.7 MiB/s wr, 530 op/s 2024-03-26T18:12:15.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 590.0 s, 492.2 tps, lat 203.299 ms stddev 229.865 2024-03-26T18:12:15.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:15 smithi115 ceph-mon[36174]: pgmap v314: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.9 MiB/s wr, 542 op/s 2024-03-26T18:12:15.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:15 smithi049 ceph-mon[32331]: pgmap v314: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.9 MiB/s wr, 542 op/s 2024-03-26T18:12:15.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:15 smithi171 ceph-mon[36033]: pgmap v314: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.9 MiB/s wr, 542 op/s 2024-03-26T18:12:17.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:17 smithi115 ceph-mon[36174]: pgmap v315: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.6 MiB/s wr, 536 op/s 2024-03-26T18:12:17.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:17 smithi049 ceph-mon[32331]: pgmap v315: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.6 MiB/s wr, 536 op/s 2024-03-26T18:12:17.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:17 smithi171 ceph-mon[36033]: pgmap v315: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.6 MiB/s wr, 536 op/s 2024-03-26T18:12:18.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:18 smithi115 ceph-mon[36174]: osdmap e103: 12 total, 12 up, 12 in 2024-03-26T18:12:18.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:18 smithi049 ceph-mon[32331]: osdmap e103: 12 total, 12 up, 12 in 2024-03-26T18:12:19.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:18 smithi171 ceph-mon[36033]: osdmap e103: 12 total, 12 up, 12 in 2024-03-26T18:12:19.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:19 smithi115 ceph-mon[36174]: pgmap v317: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.7 MiB/s wr, 548 op/s 2024-03-26T18:12:19.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:19 smithi049 ceph-mon[32331]: pgmap v317: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.7 MiB/s wr, 548 op/s 2024-03-26T18:12:20.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:19 smithi171 ceph-mon[36033]: pgmap v317: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.7 MiB/s wr, 548 op/s 2024-03-26T18:12:20.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 595.0 s, 477.8 tps, lat 203.044 ms stddev 233.565 2024-03-26T18:12:21.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:21 smithi115 ceph-mon[36174]: pgmap v318: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.7 MiB/s wr, 563 op/s 2024-03-26T18:12:21.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:21 smithi049 ceph-mon[32331]: pgmap v318: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.7 MiB/s wr, 563 op/s 2024-03-26T18:12:22.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:21 smithi171 ceph-mon[36033]: pgmap v318: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.7 MiB/s wr, 563 op/s 2024-03-26T18:12:23.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:23 smithi115 ceph-mon[36174]: pgmap v319: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 565 op/s 2024-03-26T18:12:23.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:23 smithi049 ceph-mon[32331]: pgmap v319: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 565 op/s 2024-03-26T18:12:24.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:23 smithi171 ceph-mon[36033]: pgmap v319: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 565 op/s 2024-03-26T18:12:25.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 600.0 s, 457.4 tps, lat 222.241 ms stddev 265.437 2024-03-26T18:12:25.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:25 smithi171 ceph-mon[36033]: pgmap v320: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 538 op/s 2024-03-26T18:12:25.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:25 smithi115 ceph-mon[36174]: pgmap v320: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 538 op/s 2024-03-26T18:12:25.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:25 smithi049 ceph-mon[32331]: pgmap v320: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 538 op/s 2024-03-26T18:12:27.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:27 smithi115 ceph-mon[36174]: pgmap v321: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.6 MiB/s wr, 554 op/s 2024-03-26T18:12:27.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:27 smithi049 ceph-mon[32331]: pgmap v321: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.6 MiB/s wr, 554 op/s 2024-03-26T18:12:28.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:27 smithi171 ceph-mon[36033]: pgmap v321: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.6 MiB/s wr, 554 op/s 2024-03-26T18:12:29.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:29 smithi115 ceph-mon[36174]: pgmap v322: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.2 MiB/s wr, 512 op/s 2024-03-26T18:12:29.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:29 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:12:29.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:29 smithi049 ceph-mon[32331]: pgmap v322: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.2 MiB/s wr, 512 op/s 2024-03-26T18:12:29.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:29 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:12:30.070 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:29 smithi171 ceph-mon[36033]: pgmap v322: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.2 MiB/s wr, 512 op/s 2024-03-26T18:12:30.071 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:29 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:12:30.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 605.0 s, 451.5 tps, lat 220.570 ms stddev 256.217 2024-03-26T18:12:31.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:31 smithi049 ceph-mon[32331]: pgmap v323: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.3 MiB/s wr, 564 op/s 2024-03-26T18:12:31.747 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:31.747 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:31.911 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:31 smithi171 ceph-mon[36033]: pgmap v323: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.3 MiB/s wr, 564 op/s 2024-03-26T18:12:31.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:31.912 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:31.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:31 smithi115 ceph-mon[36174]: pgmap v323: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.3 MiB/s wr, 564 op/s 2024-03-26T18:12:31.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:31.985 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:32.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:32 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:32.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:32 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:32.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:32 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:32.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:32 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:32.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:32 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:32.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:32 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:32.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:32 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:32.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:32 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:32.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:32 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:32.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:32 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:32.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:32 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:32.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:32 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:33.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:32 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:32 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:32 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:32 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:32 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:33.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:32 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:33.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:33 smithi115 ceph-mon[36174]: pgmap v324: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.0 MiB/s wr, 499 op/s 2024-03-26T18:12:33.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:33 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-03-26T18:12:33.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:33 smithi115 ceph-mon[36174]: Unable to set osd_memory_target on smithi115 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-26T18:12:33.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:33 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:33 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:33 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.982 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:33 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:33 smithi049 ceph-mon[32331]: pgmap v324: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.0 MiB/s wr, 499 op/s 2024-03-26T18:12:33.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:33 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-03-26T18:12:33.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:33 smithi049 ceph-mon[32331]: Unable to set osd_memory_target on smithi115 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-26T18:12:33.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:33 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:33 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:33 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:33.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:33 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:34.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:33 smithi171 ceph-mon[36033]: pgmap v324: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.0 MiB/s wr, 499 op/s 2024-03-26T18:12:34.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:33 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi115 to 174.1M 2024-03-26T18:12:34.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:33 smithi171 ceph-mon[36033]: Unable to set osd_memory_target on smithi115 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-26T18:12:34.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:33 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:34.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:33 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:34.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:33 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:34.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:33 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:34.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:34 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-26T18:12:34.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:34 smithi115 ceph-mon[36174]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-26T18:12:34.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:34 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-26T18:12:34.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:34 smithi049 ceph-mon[32331]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-26T18:12:35.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:34 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi049 to 176.2M 2024-03-26T18:12:35.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:34 smithi171 ceph-mon[36033]: Unable to set osd_memory_target on smithi049 to 184762368: error parsing value: Value '184762368' is below minimum 939524096 2024-03-26T18:12:35.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 610.0 s, 414.3 tps, lat 241.626 ms stddev 298.695 2024-03-26T18:12:35.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:35 smithi115 ceph-mon[36174]: pgmap v325: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.1 MiB/s wr, 507 op/s 2024-03-26T18:12:35.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:35 smithi049 ceph-mon[32331]: pgmap v325: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.1 MiB/s wr, 507 op/s 2024-03-26T18:12:36.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:35 smithi171 ceph-mon[36033]: pgmap v325: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.1 MiB/s wr, 507 op/s 2024-03-26T18:12:37.350 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:37 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:37.350 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:37 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:37.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:37 smithi171 ceph-mon[36033]: pgmap v326: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.4 MiB/s wr, 470 op/s 2024-03-26T18:12:37.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:37 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:37.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:37 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:37.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:37 smithi115 ceph-mon[36174]: pgmap v326: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.4 MiB/s wr, 470 op/s 2024-03-26T18:12:37.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:37 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:37.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:37 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:37.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:37 smithi049 ceph-mon[32331]: pgmap v326: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.4 MiB/s wr, 470 op/s 2024-03-26T18:12:38.350 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:38 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:38 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:38 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:38 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:38 smithi171 ceph-mon[36033]: Adjusting osd_memory_target on smithi171 to 174.1M 2024-03-26T18:12:38.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:38 smithi171 ceph-mon[36033]: Unable to set osd_memory_target on smithi171 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-26T18:12:38.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:38 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:12:38.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:38 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:12:38.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:38 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:38.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:38 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:38 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:38 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:38 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:38 smithi115 ceph-mon[36174]: Adjusting osd_memory_target on smithi171 to 174.1M 2024-03-26T18:12:38.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:38 smithi115 ceph-mon[36174]: Unable to set osd_memory_target on smithi171 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-26T18:12:38.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:38 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:12:38.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:38 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:12:38.482 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:38 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:38.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:38 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:38 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:38 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:38 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T18:12:38.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:38 smithi049 ceph-mon[32331]: Adjusting osd_memory_target on smithi171 to 174.1M 2024-03-26T18:12:38.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:38 smithi049 ceph-mon[32331]: Unable to set osd_memory_target on smithi171 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-26T18:12:38.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:38 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:12:38.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:38 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:12:38.497 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:38 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:12:39.350 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:39 smithi171 ceph-mon[36033]: pgmap v327: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.1 MiB/s wr, 433 op/s 2024-03-26T18:12:39.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:39 smithi115 ceph-mon[36174]: pgmap v327: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.1 MiB/s wr, 433 op/s 2024-03-26T18:12:39.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:39 smithi049 ceph-mon[32331]: pgmap v327: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.1 MiB/s wr, 433 op/s 2024-03-26T18:12:40.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 615.0 s, 494.0 tps, lat 204.434 ms stddev 247.918 2024-03-26T18:12:41.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:41 smithi115 ceph-mon[36174]: pgmap v328: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.8 MiB/s wr, 487 op/s 2024-03-26T18:12:41.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:41 smithi049 ceph-mon[32331]: pgmap v328: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.8 MiB/s wr, 487 op/s 2024-03-26T18:12:41.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:41 smithi171 ceph-mon[36033]: pgmap v328: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.8 MiB/s wr, 487 op/s 2024-03-26T18:12:43.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:43 smithi115 ceph-mon[36174]: pgmap v329: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.2 MiB/s wr, 424 op/s 2024-03-26T18:12:43.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:43 smithi049 ceph-mon[32331]: pgmap v329: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.2 MiB/s wr, 424 op/s 2024-03-26T18:12:43.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:43 smithi171 ceph-mon[36033]: pgmap v329: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.2 MiB/s wr, 424 op/s 2024-03-26T18:12:45.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 620.0 s, 498.2 tps, lat 201.971 ms stddev 263.214 2024-03-26T18:12:45.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:45 smithi049 ceph-mon[32331]: pgmap v330: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 MiB/s wr, 434 op/s 2024-03-26T18:12:45.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:45 smithi171 ceph-mon[36033]: pgmap v330: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 MiB/s wr, 434 op/s 2024-03-26T18:12:45.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:45 smithi115 ceph-mon[36174]: pgmap v330: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 MiB/s wr, 434 op/s 2024-03-26T18:12:47.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:47 smithi049 ceph-mon[32331]: pgmap v331: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 416 op/s 2024-03-26T18:12:47.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:47 smithi171 ceph-mon[36033]: pgmap v331: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 416 op/s 2024-03-26T18:12:47.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:47 smithi115 ceph-mon[36174]: pgmap v331: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 416 op/s 2024-03-26T18:12:49.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:49 smithi049 ceph-mon[32331]: pgmap v332: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.8 MiB/s wr, 406 op/s 2024-03-26T18:12:49.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:49 smithi171 ceph-mon[36033]: pgmap v332: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.8 MiB/s wr, 406 op/s 2024-03-26T18:12:49.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:49 smithi115 ceph-mon[36174]: pgmap v332: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.8 MiB/s wr, 406 op/s 2024-03-26T18:12:50.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 625.0 s, 507.4 tps, lat 196.230 ms stddev 243.628 2024-03-26T18:12:51.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:51 smithi049 ceph-mon[32331]: pgmap v333: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.0 MiB/s wr, 477 op/s 2024-03-26T18:12:51.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:51 smithi171 ceph-mon[36033]: pgmap v333: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.0 MiB/s wr, 477 op/s 2024-03-26T18:12:51.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:51 smithi115 ceph-mon[36174]: pgmap v333: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.0 MiB/s wr, 477 op/s 2024-03-26T18:12:53.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:53 smithi171 ceph-mon[36033]: pgmap v334: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 406 op/s 2024-03-26T18:12:53.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:53 smithi115 ceph-mon[36174]: pgmap v334: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 406 op/s 2024-03-26T18:12:53.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:53 smithi049 ceph-mon[32331]: pgmap v334: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 406 op/s 2024-03-26T18:12:55.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 630.0 s, 481.2 tps, lat 209.551 ms stddev 277.418 2024-03-26T18:12:55.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:55 smithi171 ceph-mon[36033]: pgmap v335: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.5 MiB/s wr, 448 op/s 2024-03-26T18:12:55.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:55 smithi115 ceph-mon[36174]: pgmap v335: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.5 MiB/s wr, 448 op/s 2024-03-26T18:12:55.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:55 smithi049 ceph-mon[32331]: pgmap v335: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.5 MiB/s wr, 448 op/s 2024-03-26T18:12:57.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:57 smithi171 ceph-mon[36033]: pgmap v336: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.2 MiB/s wr, 448 op/s 2024-03-26T18:12:57.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:57 smithi115 ceph-mon[36174]: pgmap v336: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.2 MiB/s wr, 448 op/s 2024-03-26T18:12:57.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:57 smithi049 ceph-mon[32331]: pgmap v336: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.2 MiB/s wr, 448 op/s 2024-03-26T18:12:59.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:12:59 smithi171 ceph-mon[36033]: pgmap v337: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 445 op/s 2024-03-26T18:12:59.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:12:59 smithi115 ceph-mon[36174]: pgmap v337: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 445 op/s 2024-03-26T18:12:59.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:12:59 smithi049 ceph-mon[32331]: pgmap v337: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 445 op/s 2024-03-26T18:13:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 635.0 s, 459.6 tps, lat 201.035 ms stddev 231.739 2024-03-26T18:13:01.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:01 smithi171 ceph-mon[36033]: pgmap v338: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.0 MiB/s wr, 522 op/s 2024-03-26T18:13:01.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:01 smithi115 ceph-mon[36174]: pgmap v338: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.0 MiB/s wr, 522 op/s 2024-03-26T18:13:01.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:01 smithi049 ceph-mon[32331]: pgmap v338: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.0 MiB/s wr, 522 op/s 2024-03-26T18:13:03.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:03 smithi171 ceph-mon[36033]: pgmap v339: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.8 MiB/s wr, 440 op/s 2024-03-26T18:13:03.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:03 smithi115 ceph-mon[36174]: pgmap v339: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.8 MiB/s wr, 440 op/s 2024-03-26T18:13:03.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:03 smithi049 ceph-mon[32331]: pgmap v339: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.8 MiB/s wr, 440 op/s 2024-03-26T18:13:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 640.0 s, 463.4 tps, lat 228.118 ms stddev 302.134 2024-03-26T18:13:05.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:05 smithi171 ceph-mon[36033]: pgmap v340: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 496 op/s 2024-03-26T18:13:05.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:05 smithi115 ceph-mon[36174]: pgmap v340: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 496 op/s 2024-03-26T18:13:05.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:05 smithi049 ceph-mon[32331]: pgmap v340: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 496 op/s 2024-03-26T18:13:07.549 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:07 smithi049 ceph-mon[32331]: pgmap v341: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.2 MiB/s wr, 482 op/s 2024-03-26T18:13:07.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:07 smithi171 ceph-mon[36033]: pgmap v341: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.2 MiB/s wr, 482 op/s 2024-03-26T18:13:07.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:07 smithi115 ceph-mon[36174]: pgmap v341: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.2 MiB/s wr, 482 op/s 2024-03-26T18:13:09.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:09 smithi171 ceph-mon[36033]: pgmap v342: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.4 MiB/s wr, 463 op/s 2024-03-26T18:13:09.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:09 smithi115 ceph-mon[36174]: pgmap v342: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.4 MiB/s wr, 463 op/s 2024-03-26T18:13:09.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:09 smithi049 ceph-mon[32331]: pgmap v342: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.4 MiB/s wr, 463 op/s 2024-03-26T18:13:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 645.0 s, 455.4 tps, lat 208.427 ms stddev 239.262 2024-03-26T18:13:10.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:10 smithi171 ceph-mon[36033]: osdmap e104: 12 total, 12 up, 12 in 2024-03-26T18:13:10.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:10 smithi115 ceph-mon[36174]: osdmap e104: 12 total, 12 up, 12 in 2024-03-26T18:13:10.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:10 smithi049 ceph-mon[32331]: osdmap e104: 12 total, 12 up, 12 in 2024-03-26T18:13:11.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:11 smithi171 ceph-mon[36033]: pgmap v344: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.8 MiB/s wr, 482 op/s 2024-03-26T18:13:11.631 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:11 smithi115 ceph-mon[36174]: pgmap v344: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.8 MiB/s wr, 482 op/s 2024-03-26T18:13:11.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:11 smithi049 ceph-mon[32331]: pgmap v344: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.8 MiB/s wr, 482 op/s 2024-03-26T18:13:13.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:13 smithi171 ceph-mon[36033]: pgmap v345: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 6.6 MiB/s wr, 486 op/s 2024-03-26T18:13:13.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:13 smithi115 ceph-mon[36174]: pgmap v345: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 6.6 MiB/s wr, 486 op/s 2024-03-26T18:13:13.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:13 smithi049 ceph-mon[32331]: pgmap v345: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 6.6 MiB/s wr, 486 op/s 2024-03-26T18:13:15.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 650.0 s, 502.6 tps, lat 210.323 ms stddev 280.793 2024-03-26T18:13:15.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:15 smithi171 ceph-mon[36033]: pgmap v346: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 6.2 MiB/s wr, 492 op/s 2024-03-26T18:13:15.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:15 smithi115 ceph-mon[36174]: pgmap v346: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 6.2 MiB/s wr, 492 op/s 2024-03-26T18:13:15.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:15 smithi049 ceph-mon[32331]: pgmap v346: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 6.2 MiB/s wr, 492 op/s 2024-03-26T18:13:17.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:17 smithi171 ceph-mon[36033]: pgmap v347: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2024-03-26T18:13:17.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:17 smithi115 ceph-mon[36174]: pgmap v347: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2024-03-26T18:13:17.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:17 smithi049 ceph-mon[32331]: pgmap v347: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2024-03-26T18:13:18.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:18 smithi115 ceph-mon[36174]: osdmap e105: 12 total, 12 up, 12 in 2024-03-26T18:13:18.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:18 smithi049 ceph-mon[32331]: osdmap e105: 12 total, 12 up, 12 in 2024-03-26T18:13:19.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:18 smithi171 ceph-mon[36033]: osdmap e105: 12 total, 12 up, 12 in 2024-03-26T18:13:19.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:19 smithi115 ceph-mon[36174]: pgmap v349: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-03-26T18:13:19.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:19 smithi049 ceph-mon[32331]: pgmap v349: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-03-26T18:13:20.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:19 smithi171 ceph-mon[36033]: pgmap v349: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-03-26T18:13:20.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 655.0 s, 494.0 tps, lat 200.858 ms stddev 254.800 2024-03-26T18:13:21.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:21 smithi115 ceph-mon[36174]: pgmap v350: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 6.7 MiB/s wr, 488 op/s 2024-03-26T18:13:21.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:21 smithi049 ceph-mon[32331]: pgmap v350: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 6.7 MiB/s wr, 488 op/s 2024-03-26T18:13:22.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:21 smithi171 ceph-mon[36033]: pgmap v350: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 6.7 MiB/s wr, 488 op/s 2024-03-26T18:13:23.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:23 smithi115 ceph-mon[36174]: pgmap v351: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 6.3 MiB/s wr, 488 op/s 2024-03-26T18:13:23.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:23 smithi049 ceph-mon[32331]: pgmap v351: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 6.3 MiB/s wr, 488 op/s 2024-03-26T18:13:24.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:23 smithi171 ceph-mon[36033]: pgmap v351: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 6.3 MiB/s wr, 488 op/s 2024-03-26T18:13:25.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 660.0 s, 450.6 tps, lat 219.874 ms stddev 291.161 2024-03-26T18:13:25.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:25 smithi115 ceph-mon[36174]: pgmap v352: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 6.5 MiB/s wr, 536 op/s 2024-03-26T18:13:25.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:25 smithi049 ceph-mon[32331]: pgmap v352: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 6.5 MiB/s wr, 536 op/s 2024-03-26T18:13:26.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:25 smithi171 ceph-mon[36033]: pgmap v352: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 6.5 MiB/s wr, 536 op/s 2024-03-26T18:13:27.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:27 smithi115 ceph-mon[36174]: pgmap v353: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.8 MiB/s wr, 549 op/s 2024-03-26T18:13:27.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:27 smithi049 ceph-mon[32331]: pgmap v353: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.8 MiB/s wr, 549 op/s 2024-03-26T18:13:28.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:27 smithi171 ceph-mon[36033]: pgmap v353: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.8 MiB/s wr, 549 op/s 2024-03-26T18:13:29.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:29 smithi115 ceph-mon[36174]: pgmap v354: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.2 MiB/s wr, 543 op/s 2024-03-26T18:13:29.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:29 smithi049 ceph-mon[32331]: pgmap v354: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.2 MiB/s wr, 543 op/s 2024-03-26T18:13:30.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:29 smithi171 ceph-mon[36033]: pgmap v354: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.2 MiB/s wr, 543 op/s 2024-03-26T18:13:30.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 665.0 s, 477.4 tps, lat 214.103 ms stddev 229.884 2024-03-26T18:13:31.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:31 smithi115 ceph-mon[36174]: pgmap v355: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 7.0 MiB/s wr, 579 op/s 2024-03-26T18:13:31.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:31 smithi049 ceph-mon[32331]: pgmap v355: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 7.0 MiB/s wr, 579 op/s 2024-03-26T18:13:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:31 smithi171 ceph-mon[36033]: pgmap v355: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 7.0 MiB/s wr, 579 op/s 2024-03-26T18:13:33.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:33 smithi115 ceph-mon[36174]: pgmap v356: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 5.3 MiB/s wr, 479 op/s 2024-03-26T18:13:33.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:33 smithi049 ceph-mon[32331]: pgmap v356: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 5.3 MiB/s wr, 479 op/s 2024-03-26T18:13:34.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:33 smithi171 ceph-mon[36033]: pgmap v356: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 5.3 MiB/s wr, 479 op/s 2024-03-26T18:13:35.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 670.0 s, 456.0 tps, lat 218.599 ms stddev 263.716 2024-03-26T18:13:35.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:35 smithi115 ceph-mon[36174]: pgmap v357: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 6.5 MiB/s wr, 555 op/s 2024-03-26T18:13:35.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:35 smithi049 ceph-mon[32331]: pgmap v357: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 6.5 MiB/s wr, 555 op/s 2024-03-26T18:13:36.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:35 smithi171 ceph-mon[36033]: pgmap v357: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 6.5 MiB/s wr, 555 op/s 2024-03-26T18:13:37.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:37 smithi115 ceph-mon[36174]: pgmap v358: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 6.9 MiB/s wr, 576 op/s 2024-03-26T18:13:37.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:37 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:13:37.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:37 smithi049 ceph-mon[32331]: pgmap v358: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 6.9 MiB/s wr, 576 op/s 2024-03-26T18:13:37.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:37 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:13:38.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:37 smithi171 ceph-mon[36033]: pgmap v358: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 6.9 MiB/s wr, 576 op/s 2024-03-26T18:13:38.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:37 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:13:39.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:39 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:39 smithi115 ceph-mon[36174]: pgmap v359: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.9 MiB/s wr, 584 op/s 2024-03-26T18:13:39.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:39 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:39 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:39 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:39 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:39 smithi049 ceph-mon[32331]: pgmap v359: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.9 MiB/s wr, 584 op/s 2024-03-26T18:13:39.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:39 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:39 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:39 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:39 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:39 smithi171 ceph-mon[36033]: pgmap v359: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.9 MiB/s wr, 584 op/s 2024-03-26T18:13:39.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:39 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:39 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:39.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:39 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:40.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 675.0 s, 412.8 tps, lat 234.055 ms stddev 247.501 2024-03-26T18:13:40.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:40 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:13:40.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:40 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:13:40.481 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:40 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:40.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:40 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:13:40.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:40 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:13:40.496 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:40 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:40.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:40 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:13:40.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:40 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:13:40.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:40 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:13:41.599 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:41 smithi171 ceph-mon[36033]: pgmap v360: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.7 MiB/s wr, 689 op/s 2024-03-26T18:13:41.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:41 smithi115 ceph-mon[36174]: pgmap v360: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.7 MiB/s wr, 689 op/s 2024-03-26T18:13:41.746 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:41 smithi049 ceph-mon[32331]: pgmap v360: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.7 MiB/s wr, 689 op/s 2024-03-26T18:13:43.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:43 smithi115 ceph-mon[36174]: pgmap v361: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.3 MiB/s wr, 597 op/s 2024-03-26T18:13:43.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:43 smithi049 ceph-mon[32331]: pgmap v361: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.3 MiB/s wr, 597 op/s 2024-03-26T18:13:44.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:43 smithi171 ceph-mon[36033]: pgmap v361: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.3 MiB/s wr, 597 op/s 2024-03-26T18:13:45.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 680.0 s, 484.8 tps, lat 218.872 ms stddev 277.168 2024-03-26T18:13:45.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:45 smithi115 ceph-mon[36174]: pgmap v362: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.8 MiB/s wr, 677 op/s 2024-03-26T18:13:45.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:45 smithi049 ceph-mon[32331]: pgmap v362: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.8 MiB/s wr, 677 op/s 2024-03-26T18:13:46.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:45 smithi171 ceph-mon[36033]: pgmap v362: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.8 MiB/s wr, 677 op/s 2024-03-26T18:13:47.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:47 smithi115 ceph-mon[36174]: pgmap v363: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.7 MiB/s wr, 704 op/s 2024-03-26T18:13:47.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:47 smithi049 ceph-mon[32331]: pgmap v363: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.7 MiB/s wr, 704 op/s 2024-03-26T18:13:48.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:47 smithi171 ceph-mon[36033]: pgmap v363: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.7 MiB/s wr, 704 op/s 2024-03-26T18:13:49.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:49 smithi115 ceph-mon[36174]: pgmap v364: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.9 MiB/s wr, 637 op/s 2024-03-26T18:13:49.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:49 smithi049 ceph-mon[32331]: pgmap v364: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.9 MiB/s wr, 637 op/s 2024-03-26T18:13:50.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:49 smithi171 ceph-mon[36033]: pgmap v364: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.9 MiB/s wr, 637 op/s 2024-03-26T18:13:50.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 685.0 s, 517.4 tps, lat 186.888 ms stddev 202.483 2024-03-26T18:13:51.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:51 smithi115 ceph-mon[36174]: pgmap v365: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.5 MiB/s wr, 692 op/s 2024-03-26T18:13:51.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:51 smithi049 ceph-mon[32331]: pgmap v365: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.5 MiB/s wr, 692 op/s 2024-03-26T18:13:52.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:51 smithi171 ceph-mon[36033]: pgmap v365: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.5 MiB/s wr, 692 op/s 2024-03-26T18:13:53.981 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:53 smithi115 ceph-mon[36174]: pgmap v366: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.4 MiB/s wr, 577 op/s 2024-03-26T18:13:53.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:53 smithi049 ceph-mon[32331]: pgmap v366: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.4 MiB/s wr, 577 op/s 2024-03-26T18:13:54.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:53 smithi171 ceph-mon[36033]: pgmap v366: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.4 MiB/s wr, 577 op/s 2024-03-26T18:13:55.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 690.0 s, 497.4 tps, lat 200.587 ms stddev 229.114 2024-03-26T18:13:55.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:55 smithi115 ceph-mon[36174]: pgmap v367: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.8 MiB/s wr, 670 op/s 2024-03-26T18:13:55.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:55 smithi049 ceph-mon[32331]: pgmap v367: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.8 MiB/s wr, 670 op/s 2024-03-26T18:13:56.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:55 smithi171 ceph-mon[36033]: pgmap v367: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.8 MiB/s wr, 670 op/s 2024-03-26T18:13:57.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:57 smithi115 ceph-mon[36174]: pgmap v368: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 641 op/s 2024-03-26T18:13:57.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:57 smithi049 ceph-mon[32331]: pgmap v368: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 641 op/s 2024-03-26T18:13:58.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:57 smithi171 ceph-mon[36033]: pgmap v368: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 641 op/s 2024-03-26T18:13:59.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:13:59 smithi115 ceph-mon[36174]: pgmap v369: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 585 op/s 2024-03-26T18:13:59.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:13:59 smithi049 ceph-mon[32331]: pgmap v369: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 585 op/s 2024-03-26T18:14:00.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:13:59 smithi171 ceph-mon[36033]: pgmap v369: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 585 op/s 2024-03-26T18:14:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 695.0 s, 500.8 tps, lat 200.049 ms stddev 222.854 2024-03-26T18:14:01.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:01 smithi115 ceph-mon[36174]: pgmap v370: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-03-26T18:14:01.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:01 smithi049 ceph-mon[32331]: pgmap v370: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-03-26T18:14:02.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:01 smithi171 ceph-mon[36033]: pgmap v370: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-03-26T18:14:03.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:03 smithi115 ceph-mon[36174]: pgmap v371: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.8 MiB/s wr, 493 op/s 2024-03-26T18:14:03.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:03 smithi049 ceph-mon[32331]: pgmap v371: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.8 MiB/s wr, 493 op/s 2024-03-26T18:14:04.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:03 smithi171 ceph-mon[36033]: pgmap v371: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.8 MiB/s wr, 493 op/s 2024-03-26T18:14:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 700.0 s, 300.0 tps, lat 325.671 ms stddev 306.324 2024-03-26T18:14:05.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:05 smithi115 ceph-mon[36174]: pgmap v372: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 508 op/s 2024-03-26T18:14:05.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:05 smithi049 ceph-mon[32331]: pgmap v372: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 508 op/s 2024-03-26T18:14:06.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:05 smithi171 ceph-mon[36033]: pgmap v372: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 508 op/s 2024-03-26T18:14:07.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:07 smithi115 ceph-mon[36174]: pgmap v373: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.6 MiB/s wr, 510 op/s 2024-03-26T18:14:07.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:07 smithi049 ceph-mon[32331]: pgmap v373: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.6 MiB/s wr, 510 op/s 2024-03-26T18:14:08.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:07 smithi171 ceph-mon[36033]: pgmap v373: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.6 MiB/s wr, 510 op/s 2024-03-26T18:14:09.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:09 smithi115 ceph-mon[36174]: pgmap v374: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.3 MiB/s wr, 534 op/s 2024-03-26T18:14:09.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:09 smithi115 ceph-mon[36174]: osdmap e106: 12 total, 12 up, 12 in 2024-03-26T18:14:09.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:09 smithi049 ceph-mon[32331]: pgmap v374: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.3 MiB/s wr, 534 op/s 2024-03-26T18:14:09.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:09 smithi049 ceph-mon[32331]: osdmap e106: 12 total, 12 up, 12 in 2024-03-26T18:14:10.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:09 smithi171 ceph-mon[36033]: pgmap v374: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.3 MiB/s wr, 534 op/s 2024-03-26T18:14:10.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:09 smithi171 ceph-mon[36033]: osdmap e106: 12 total, 12 up, 12 in 2024-03-26T18:14:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 705.0 s, 141.2 tps, lat 672.014 ms stddev 754.841 2024-03-26T18:14:11.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:11 smithi115 ceph-mon[36174]: pgmap v376: 129 pgs: 27 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 94 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-03-26T18:14:11.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:11 smithi049 ceph-mon[32331]: pgmap v376: 129 pgs: 27 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 94 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-03-26T18:14:12.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:11 smithi171 ceph-mon[36033]: pgmap v376: 129 pgs: 27 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 94 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 1.13k op/s 2024-03-26T18:14:13.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:13 smithi115 ceph-mon[36174]: pgmap v377: 129 pgs: 27 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 94 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-03-26T18:14:13.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:13 smithi049 ceph-mon[32331]: pgmap v377: 129 pgs: 27 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 94 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-03-26T18:14:14.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:13 smithi171 ceph-mon[36033]: pgmap v377: 129 pgs: 27 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 94 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-03-26T18:14:15.204 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 710.0 s, 551.6 tps, lat 192.396 ms stddev 331.428 2024-03-26T18:14:15.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:15 smithi049 ceph-mon[32331]: pgmap v378: 129 pgs: 27 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 94 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-03-26T18:14:16.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:15 smithi171 ceph-mon[36033]: pgmap v378: 129 pgs: 27 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 94 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-03-26T18:14:16.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:15 smithi115 ceph-mon[36174]: pgmap v378: 129 pgs: 27 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 94 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2024-03-26T18:14:17.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:17 smithi049 ceph-mon[32331]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-03-26T18:14:17.996 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:17 smithi049 ceph-mon[32331]: osdmap e107: 12 total, 12 up, 12 in 2024-03-26T18:14:18.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:17 smithi171 ceph-mon[36033]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-03-26T18:14:18.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:17 smithi171 ceph-mon[36033]: osdmap e107: 12 total, 12 up, 12 in 2024-03-26T18:14:18.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:17 smithi115 ceph-mon[36174]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-03-26T18:14:18.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:17 smithi115 ceph-mon[36174]: osdmap e107: 12 total, 12 up, 12 in 2024-03-26T18:14:19.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:19 smithi049 ceph-mon[32331]: pgmap v381: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.3 MiB/s wr, 746 op/s 2024-03-26T18:14:20.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:19 smithi171 ceph-mon[36033]: pgmap v381: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.3 MiB/s wr, 746 op/s 2024-03-26T18:14:20.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 715.0 s, 521.6 tps, lat 196.488 ms stddev 234.654 2024-03-26T18:14:20.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:19 smithi115 ceph-mon[36174]: pgmap v381: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.3 MiB/s wr, 746 op/s 2024-03-26T18:14:21.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:21 smithi049 ceph-mon[32331]: pgmap v382: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 482 op/s 2024-03-26T18:14:22.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:21 smithi171 ceph-mon[36033]: pgmap v382: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 482 op/s 2024-03-26T18:14:22.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:21 smithi115 ceph-mon[36174]: pgmap v382: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 482 op/s 2024-03-26T18:14:24.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:23 smithi171 ceph-mon[36033]: pgmap v383: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.1 MiB/s wr, 458 op/s 2024-03-26T18:14:24.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:23 smithi115 ceph-mon[36174]: pgmap v383: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.1 MiB/s wr, 458 op/s 2024-03-26T18:14:24.246 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:23 smithi049 ceph-mon[32331]: pgmap v383: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.1 MiB/s wr, 458 op/s 2024-03-26T18:14:25.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 720.0 s, 486.6 tps, lat 202.699 ms stddev 242.713 2024-03-26T18:14:26.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:25 smithi171 ceph-mon[36033]: pgmap v384: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.0 MiB/s wr, 479 op/s 2024-03-26T18:14:26.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:25 smithi115 ceph-mon[36174]: pgmap v384: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.0 MiB/s wr, 479 op/s 2024-03-26T18:14:26.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:25 smithi049 ceph-mon[32331]: pgmap v384: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.0 MiB/s wr, 479 op/s 2024-03-26T18:14:28.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:27 smithi171 ceph-mon[36033]: pgmap v385: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.6 MiB/s wr, 490 op/s 2024-03-26T18:14:28.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:27 smithi115 ceph-mon[36174]: pgmap v385: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.6 MiB/s wr, 490 op/s 2024-03-26T18:14:28.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:27 smithi049 ceph-mon[32331]: pgmap v385: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.6 MiB/s wr, 490 op/s 2024-03-26T18:14:29.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:28 smithi171 ceph-mon[36033]: pgmap v386: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 5.5 MiB/s wr, 488 op/s 2024-03-26T18:14:29.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:28 smithi115 ceph-mon[36174]: pgmap v386: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 5.5 MiB/s wr, 488 op/s 2024-03-26T18:14:29.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:28 smithi049 ceph-mon[32331]: pgmap v386: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 5.5 MiB/s wr, 488 op/s 2024-03-26T18:14:30.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 725.0 s, 467.4 tps, lat 214.602 ms stddev 267.602 2024-03-26T18:14:31.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:31 smithi171 ceph-mon[36033]: pgmap v387: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 6.3 MiB/s wr, 560 op/s 2024-03-26T18:14:31.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:31 smithi115 ceph-mon[36174]: pgmap v387: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 6.3 MiB/s wr, 560 op/s 2024-03-26T18:14:31.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:31 smithi049 ceph-mon[32331]: pgmap v387: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 6.3 MiB/s wr, 560 op/s 2024-03-26T18:14:33.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:33 smithi171 ceph-mon[36033]: pgmap v388: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 4.9 MiB/s wr, 457 op/s 2024-03-26T18:14:33.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:33 smithi115 ceph-mon[36174]: pgmap v388: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 4.9 MiB/s wr, 457 op/s 2024-03-26T18:14:33.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:33 smithi049 ceph-mon[32331]: pgmap v388: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 4.9 MiB/s wr, 457 op/s 2024-03-26T18:14:35.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 730.0 s, 494.0 tps, lat 209.384 ms stddev 239.575 2024-03-26T18:14:35.599 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:35 smithi171 ceph-mon[36033]: pgmap v389: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 5.4 MiB/s wr, 480 op/s 2024-03-26T18:14:35.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:35 smithi115 ceph-mon[36174]: pgmap v389: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 5.4 MiB/s wr, 480 op/s 2024-03-26T18:14:35.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:35 smithi049 ceph-mon[32331]: pgmap v389: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 5.4 MiB/s wr, 480 op/s 2024-03-26T18:14:37.547 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:37 smithi049 ceph-mon[32331]: pgmap v390: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 5.3 MiB/s wr, 490 op/s 2024-03-26T18:14:37.600 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:37 smithi171 ceph-mon[36033]: pgmap v390: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 5.3 MiB/s wr, 490 op/s 2024-03-26T18:14:37.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:37 smithi115 ceph-mon[36174]: pgmap v390: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 5.3 MiB/s wr, 490 op/s 2024-03-26T18:14:39.599 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:39 smithi171 ceph-mon[36033]: pgmap v391: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 5.3 MiB/s wr, 488 op/s 2024-03-26T18:14:39.733 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:39 smithi115 ceph-mon[36174]: pgmap v391: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 5.3 MiB/s wr, 488 op/s 2024-03-26T18:14:39.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:39 smithi049 ceph-mon[32331]: pgmap v391: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 5.3 MiB/s wr, 488 op/s 2024-03-26T18:14:40.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 735.0 s, 487.4 tps, lat 193.397 ms stddev 213.583 2024-03-26T18:14:40.599 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:40 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:14:40.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:40 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:14:40.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:40 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:14:41.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:41 smithi115 ceph-mon[36174]: pgmap v392: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-03-26T18:14:41.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:41 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:14:41.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:41 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:14:41.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:41 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:14:41.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:41 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:14:41.731 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:41 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:14:41.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:41 smithi049 ceph-mon[32331]: pgmap v392: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-03-26T18:14:41.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:41 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:14:41.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:41 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:14:41.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:41 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:14:41.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:41 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:14:41.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:41 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:14:41.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:41 smithi171 ceph-mon[36033]: pgmap v392: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2024-03-26T18:14:41.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:41 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:14:41.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:41 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:14:41.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:41 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:14:41.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:41 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:14:41.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:41 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:14:43.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:43 smithi115 ceph-mon[36174]: pgmap v393: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2024-03-26T18:14:43.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:43 smithi049 ceph-mon[32331]: pgmap v393: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2024-03-26T18:14:43.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:43 smithi171 ceph-mon[36033]: pgmap v393: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2024-03-26T18:14:45.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 740.0 s, 490.8 tps, lat 208.221 ms stddev 236.472 2024-03-26T18:14:45.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:45 smithi115 ceph-mon[36174]: pgmap v394: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.8 MiB/s wr, 546 op/s 2024-03-26T18:14:45.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:45 smithi049 ceph-mon[32331]: pgmap v394: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.8 MiB/s wr, 546 op/s 2024-03-26T18:14:45.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:45 smithi171 ceph-mon[36033]: pgmap v394: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.8 MiB/s wr, 546 op/s 2024-03-26T18:14:47.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:47 smithi115 ceph-mon[36174]: pgmap v395: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 560 op/s 2024-03-26T18:14:47.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:47 smithi049 ceph-mon[32331]: pgmap v395: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 560 op/s 2024-03-26T18:14:47.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:47 smithi171 ceph-mon[36033]: pgmap v395: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 560 op/s 2024-03-26T18:14:49.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:49 smithi049 ceph-mon[32331]: pgmap v396: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2024-03-26T18:14:49.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:49 smithi171 ceph-mon[36033]: pgmap v396: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2024-03-26T18:14:49.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:49 smithi115 ceph-mon[36174]: pgmap v396: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2024-03-26T18:14:50.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 745.0 s, 445.8 tps, lat 229.290 ms stddev 272.958 2024-03-26T18:14:51.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:51 smithi171 ceph-mon[36033]: pgmap v397: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.7 MiB/s wr, 607 op/s 2024-03-26T18:14:51.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:51 smithi115 ceph-mon[36174]: pgmap v397: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.7 MiB/s wr, 607 op/s 2024-03-26T18:14:51.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:51 smithi049 ceph-mon[32331]: pgmap v397: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.7 MiB/s wr, 607 op/s 2024-03-26T18:14:53.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:53 smithi171 ceph-mon[36033]: pgmap v398: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.0 MiB/s wr, 451 op/s 2024-03-26T18:14:53.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:53 smithi115 ceph-mon[36174]: pgmap v398: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.0 MiB/s wr, 451 op/s 2024-03-26T18:14:53.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:53 smithi049 ceph-mon[32331]: pgmap v398: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.0 MiB/s wr, 451 op/s 2024-03-26T18:14:55.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 750.0 s, 505.0 tps, lat 198.178 ms stddev 229.226 2024-03-26T18:14:55.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:55 smithi171 ceph-mon[36033]: pgmap v399: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.6 MiB/s wr, 434 op/s 2024-03-26T18:14:55.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:55 smithi115 ceph-mon[36174]: pgmap v399: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.6 MiB/s wr, 434 op/s 2024-03-26T18:14:55.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:55 smithi049 ceph-mon[32331]: pgmap v399: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.6 MiB/s wr, 434 op/s 2024-03-26T18:14:57.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:57 smithi171 ceph-mon[36033]: pgmap v400: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 5.7 MiB/s wr, 500 op/s 2024-03-26T18:14:57.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:57 smithi115 ceph-mon[36174]: pgmap v400: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 5.7 MiB/s wr, 500 op/s 2024-03-26T18:14:57.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:57 smithi049 ceph-mon[32331]: pgmap v400: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 5.7 MiB/s wr, 500 op/s 2024-03-26T18:14:59.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:14:59 smithi171 ceph-mon[36033]: pgmap v401: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 5.1 MiB/s wr, 477 op/s 2024-03-26T18:14:59.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:14:59 smithi115 ceph-mon[36174]: pgmap v401: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 5.1 MiB/s wr, 477 op/s 2024-03-26T18:14:59.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:14:59 smithi049 ceph-mon[32331]: pgmap v401: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 5.1 MiB/s wr, 477 op/s 2024-03-26T18:15:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 755.0 s, 499.8 tps, lat 200.731 ms stddev 259.257 2024-03-26T18:15:01.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:01 smithi171 ceph-mon[36033]: pgmap v402: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 5.7 MiB/s wr, 552 op/s 2024-03-26T18:15:01.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:01 smithi115 ceph-mon[36174]: pgmap v402: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 5.7 MiB/s wr, 552 op/s 2024-03-26T18:15:01.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:01 smithi049 ceph-mon[32331]: pgmap v402: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 5.7 MiB/s wr, 552 op/s 2024-03-26T18:15:03.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:03 smithi171 ceph-mon[36033]: pgmap v403: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 5.0 MiB/s wr, 479 op/s 2024-03-26T18:15:03.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:03 smithi115 ceph-mon[36174]: pgmap v403: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 5.0 MiB/s wr, 479 op/s 2024-03-26T18:15:03.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:03 smithi049 ceph-mon[32331]: pgmap v403: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 5.0 MiB/s wr, 479 op/s 2024-03-26T18:15:05.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 760.0 s, 431.8 tps, lat 226.938 ms stddev 277.634 2024-03-26T18:15:05.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:05 smithi171 ceph-mon[36033]: pgmap v404: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-03-26T18:15:05.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:05 smithi115 ceph-mon[36174]: pgmap v404: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-03-26T18:15:05.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:05 smithi049 ceph-mon[32331]: pgmap v404: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-03-26T18:15:07.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:07 smithi171 ceph-mon[36033]: pgmap v405: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.4 MiB/s wr, 520 op/s 2024-03-26T18:15:07.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:07 smithi115 ceph-mon[36174]: pgmap v405: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.4 MiB/s wr, 520 op/s 2024-03-26T18:15:07.997 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:07 smithi049 ceph-mon[32331]: pgmap v405: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.4 MiB/s wr, 520 op/s 2024-03-26T18:15:09.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:09 smithi171 ceph-mon[36033]: pgmap v406: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 4.6 MiB/s wr, 452 op/s 2024-03-26T18:15:09.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:09 smithi171 ceph-mon[36033]: osdmap e108: 12 total, 12 up, 12 in 2024-03-26T18:15:09.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:09 smithi115 ceph-mon[36174]: pgmap v406: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 4.6 MiB/s wr, 452 op/s 2024-03-26T18:15:09.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:09 smithi115 ceph-mon[36174]: osdmap e108: 12 total, 12 up, 12 in 2024-03-26T18:15:09.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:09 smithi049 ceph-mon[32331]: pgmap v406: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 4.6 MiB/s wr, 452 op/s 2024-03-26T18:15:09.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:09 smithi049 ceph-mon[32331]: osdmap e108: 12 total, 12 up, 12 in 2024-03-26T18:15:10.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 765.0 s, 366.4 tps, lat 261.881 ms stddev 332.322 2024-03-26T18:15:11.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:11 smithi171 ceph-mon[36033]: pgmap v408: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-03-26T18:15:11.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:11 smithi115 ceph-mon[36174]: pgmap v408: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-03-26T18:15:11.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:11 smithi049 ceph-mon[32331]: pgmap v408: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 4.9 MiB/s wr, 458 op/s 2024-03-26T18:15:13.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:13 smithi171 ceph-mon[36033]: pgmap v409: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 4.0 MiB/s wr, 401 op/s 2024-03-26T18:15:13.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:13 smithi115 ceph-mon[36174]: pgmap v409: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 4.0 MiB/s wr, 401 op/s 2024-03-26T18:15:13.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:13 smithi049 ceph-mon[32331]: pgmap v409: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 4.0 MiB/s wr, 401 op/s 2024-03-26T18:15:15.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 770.0 s, 437.8 tps, lat 236.493 ms stddev 284.293 2024-03-26T18:15:15.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:15 smithi171 ceph-mon[36033]: pgmap v410: 129 pgs: 2 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 118 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 4.5 MiB/s wr, 423 op/s 2024-03-26T18:15:15.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:15 smithi115 ceph-mon[36174]: pgmap v410: 129 pgs: 2 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 118 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 4.5 MiB/s wr, 423 op/s 2024-03-26T18:15:15.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:15 smithi049 ceph-mon[32331]: pgmap v410: 129 pgs: 2 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 118 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 4.5 MiB/s wr, 423 op/s 2024-03-26T18:15:17.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:17 smithi171 ceph-mon[36033]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.4 MiB/s wr, 427 op/s 2024-03-26T18:15:17.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:17 smithi115 ceph-mon[36174]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.4 MiB/s wr, 427 op/s 2024-03-26T18:15:17.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:17 smithi049 ceph-mon[32331]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 4.4 MiB/s wr, 427 op/s 2024-03-26T18:15:18.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:18 smithi115 ceph-mon[36174]: osdmap e109: 12 total, 12 up, 12 in 2024-03-26T18:15:18.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:18 smithi049 ceph-mon[32331]: osdmap e109: 12 total, 12 up, 12 in 2024-03-26T18:15:19.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:18 smithi171 ceph-mon[36033]: osdmap e109: 12 total, 12 up, 12 in 2024-03-26T18:15:19.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:19 smithi115 ceph-mon[36174]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 4.2 MiB/s wr, 432 op/s 2024-03-26T18:15:19.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:19 smithi049 ceph-mon[32331]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 4.2 MiB/s wr, 432 op/s 2024-03-26T18:15:20.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:19 smithi171 ceph-mon[36033]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 4.2 MiB/s wr, 432 op/s 2024-03-26T18:15:20.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 775.0 s, 450.4 tps, lat 219.407 ms stddev 258.766 2024-03-26T18:15:21.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:21 smithi115 ceph-mon[36174]: pgmap v414: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.0 MiB/s wr, 421 op/s 2024-03-26T18:15:21.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:21 smithi049 ceph-mon[32331]: pgmap v414: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.0 MiB/s wr, 421 op/s 2024-03-26T18:15:22.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:21 smithi171 ceph-mon[36033]: pgmap v414: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.0 MiB/s wr, 421 op/s 2024-03-26T18:15:23.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:23 smithi115 ceph-mon[36174]: pgmap v415: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.0 MiB/s wr, 424 op/s 2024-03-26T18:15:23.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:23 smithi049 ceph-mon[32331]: pgmap v415: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.0 MiB/s wr, 424 op/s 2024-03-26T18:15:24.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:23 smithi171 ceph-mon[36033]: pgmap v415: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.0 MiB/s wr, 424 op/s 2024-03-26T18:15:25.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 780.0 s, 479.4 tps, lat 213.600 ms stddev 231.930 2024-03-26T18:15:25.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:25 smithi115 ceph-mon[36174]: pgmap v416: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-03-26T18:15:25.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:25 smithi049 ceph-mon[32331]: pgmap v416: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-03-26T18:15:26.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:25 smithi171 ceph-mon[36033]: pgmap v416: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.2 MiB/s wr, 438 op/s 2024-03-26T18:15:27.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:27 smithi115 ceph-mon[36174]: pgmap v417: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.8 MiB/s wr, 473 op/s 2024-03-26T18:15:27.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:27 smithi049 ceph-mon[32331]: pgmap v417: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.8 MiB/s wr, 473 op/s 2024-03-26T18:15:28.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:27 smithi171 ceph-mon[36033]: pgmap v417: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.8 MiB/s wr, 473 op/s 2024-03-26T18:15:29.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:29 smithi115 ceph-mon[36174]: pgmap v418: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.5 MiB/s wr, 461 op/s 2024-03-26T18:15:29.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:29 smithi049 ceph-mon[32331]: pgmap v418: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.5 MiB/s wr, 461 op/s 2024-03-26T18:15:30.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:29 smithi171 ceph-mon[36033]: pgmap v418: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.5 MiB/s wr, 461 op/s 2024-03-26T18:15:30.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 785.0 s, 391.2 tps, lat 243.705 ms stddev 277.980 2024-03-26T18:15:31.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:31 smithi115 ceph-mon[36174]: pgmap v419: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-03-26T18:15:31.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:31 smithi049 ceph-mon[32331]: pgmap v419: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-03-26T18:15:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:31 smithi171 ceph-mon[36033]: pgmap v419: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2024-03-26T18:15:33.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:33 smithi115 ceph-mon[36174]: pgmap v420: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 4.0 MiB/s wr, 399 op/s 2024-03-26T18:15:33.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:33 smithi049 ceph-mon[32331]: pgmap v420: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 4.0 MiB/s wr, 399 op/s 2024-03-26T18:15:34.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:33 smithi171 ceph-mon[36033]: pgmap v420: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 4.0 MiB/s wr, 399 op/s 2024-03-26T18:15:35.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 790.0 s, 502.6 tps, lat 207.933 ms stddev 276.509 2024-03-26T18:15:35.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:35 smithi115 ceph-mon[36174]: pgmap v421: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-03-26T18:15:35.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:35 smithi049 ceph-mon[32331]: pgmap v421: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-03-26T18:15:36.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:35 smithi171 ceph-mon[36033]: pgmap v421: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-03-26T18:15:37.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:37 smithi115 ceph-mon[36174]: pgmap v422: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-03-26T18:15:37.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:37 smithi049 ceph-mon[32331]: pgmap v422: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-03-26T18:15:38.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:37 smithi171 ceph-mon[36033]: pgmap v422: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2024-03-26T18:15:39.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:39 smithi115 ceph-mon[36174]: pgmap v423: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-03-26T18:15:39.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:39 smithi049 ceph-mon[32331]: pgmap v423: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-03-26T18:15:40.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:39 smithi171 ceph-mon[36033]: pgmap v423: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-03-26T18:15:40.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 795.0 s, 497.8 tps, lat 199.901 ms stddev 255.000 2024-03-26T18:15:41.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:41 smithi115 ceph-mon[36174]: pgmap v424: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 5.3 MiB/s wr, 533 op/s 2024-03-26T18:15:41.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:41 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:15:41.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:41 smithi049 ceph-mon[32331]: pgmap v424: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 5.3 MiB/s wr, 533 op/s 2024-03-26T18:15:41.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:41 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:15:42.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:41 smithi171 ceph-mon[36033]: pgmap v424: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 5.3 MiB/s wr, 533 op/s 2024-03-26T18:15:42.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:41 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:15:43.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:43 smithi049 ceph-mon[32331]: pgmap v425: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-03-26T18:15:43.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:43 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:15:43.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:43 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:15:43.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:43 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:15:44.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:43 smithi171 ceph-mon[36033]: pgmap v425: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-03-26T18:15:44.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:43 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:15:44.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:43 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:15:44.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:43 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:15:44.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:43 smithi115 ceph-mon[36174]: pgmap v425: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-03-26T18:15:44.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:43 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:15:44.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:43 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:15:44.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:43 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:15:45.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 800.0 s, 470.8 tps, lat 208.474 ms stddev 236.592 2024-03-26T18:15:45.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:45 smithi049 ceph-mon[32331]: pgmap v426: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-03-26T18:15:46.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:45 smithi171 ceph-mon[36033]: pgmap v426: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-03-26T18:15:46.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:45 smithi115 ceph-mon[36174]: pgmap v426: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-03-26T18:15:47.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:47 smithi049 ceph-mon[32331]: pgmap v427: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 545 op/s 2024-03-26T18:15:48.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:47 smithi171 ceph-mon[36033]: pgmap v427: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 545 op/s 2024-03-26T18:15:48.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:47 smithi115 ceph-mon[36174]: pgmap v427: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 545 op/s 2024-03-26T18:15:50.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:49 smithi171 ceph-mon[36033]: pgmap v428: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-03-26T18:15:50.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 805.0 s, 482.4 tps, lat 209.520 ms stddev 241.111 2024-03-26T18:15:50.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:49 smithi115 ceph-mon[36174]: pgmap v428: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-03-26T18:15:50.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:49 smithi049 ceph-mon[32331]: pgmap v428: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.3 MiB/s wr, 462 op/s 2024-03-26T18:15:52.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:51 smithi171 ceph-mon[36033]: pgmap v429: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-03-26T18:15:52.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:51 smithi115 ceph-mon[36174]: pgmap v429: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-03-26T18:15:52.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:51 smithi049 ceph-mon[32331]: pgmap v429: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-03-26T18:15:54.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:53 smithi171 ceph-mon[36033]: pgmap v430: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-03-26T18:15:54.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:53 smithi115 ceph-mon[36174]: pgmap v430: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-03-26T18:15:54.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:53 smithi049 ceph-mon[32331]: pgmap v430: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-03-26T18:15:55.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 810.0 s, 505.4 tps, lat 199.714 ms stddev 213.617 2024-03-26T18:15:56.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:55 smithi171 ceph-mon[36033]: pgmap v431: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.3 MiB/s wr, 448 op/s 2024-03-26T18:15:56.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:55 smithi115 ceph-mon[36174]: pgmap v431: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.3 MiB/s wr, 448 op/s 2024-03-26T18:15:56.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:55 smithi049 ceph-mon[32331]: pgmap v431: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.3 MiB/s wr, 448 op/s 2024-03-26T18:15:58.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:57 smithi171 ceph-mon[36033]: pgmap v432: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.1 MiB/s wr, 522 op/s 2024-03-26T18:15:58.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:57 smithi115 ceph-mon[36174]: pgmap v432: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.1 MiB/s wr, 522 op/s 2024-03-26T18:15:58.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:57 smithi049 ceph-mon[32331]: pgmap v432: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.1 MiB/s wr, 522 op/s 2024-03-26T18:16:00.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:15:59 smithi171 ceph-mon[36033]: pgmap v433: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-03-26T18:16:00.203 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 815.0 s, 460.0 tps, lat 214.823 ms stddev 239.329 2024-03-26T18:16:00.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:15:59 smithi115 ceph-mon[36174]: pgmap v433: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-03-26T18:16:00.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:15:59 smithi049 ceph-mon[32331]: pgmap v433: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 453 op/s 2024-03-26T18:16:02.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:01 smithi171 ceph-mon[36033]: pgmap v434: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.7 MiB/s wr, 496 op/s 2024-03-26T18:16:02.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:01 smithi115 ceph-mon[36174]: pgmap v434: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.7 MiB/s wr, 496 op/s 2024-03-26T18:16:02.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:01 smithi049 ceph-mon[32331]: pgmap v434: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.7 MiB/s wr, 496 op/s 2024-03-26T18:16:04.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:03 smithi171 ceph-mon[36033]: pgmap v435: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.4 MiB/s wr, 464 op/s 2024-03-26T18:16:04.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:03 smithi115 ceph-mon[36174]: pgmap v435: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.4 MiB/s wr, 464 op/s 2024-03-26T18:16:04.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:03 smithi049 ceph-mon[32331]: pgmap v435: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.4 MiB/s wr, 464 op/s 2024-03-26T18:16:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 820.0 s, 503.4 tps, lat 200.779 ms stddev 252.352 2024-03-26T18:16:06.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:05 smithi171 ceph-mon[36033]: pgmap v436: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-03-26T18:16:06.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:05 smithi115 ceph-mon[36174]: pgmap v436: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-03-26T18:16:06.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:05 smithi049 ceph-mon[32331]: pgmap v436: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-03-26T18:16:08.047 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:07 smithi049 ceph-mon[32331]: pgmap v437: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-03-26T18:16:08.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:07 smithi171 ceph-mon[36033]: pgmap v437: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-03-26T18:16:08.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:07 smithi115 ceph-mon[36174]: pgmap v437: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-03-26T18:16:10.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:09 smithi171 ceph-mon[36033]: pgmap v438: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.3 MiB/s wr, 451 op/s 2024-03-26T18:16:10.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:09 smithi171 ceph-mon[36033]: osdmap e110: 12 total, 12 up, 12 in 2024-03-26T18:16:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 825.0 s, 402.2 tps, lat 242.966 ms stddev 293.090 2024-03-26T18:16:10.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:09 smithi115 ceph-mon[36174]: pgmap v438: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.3 MiB/s wr, 451 op/s 2024-03-26T18:16:10.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:09 smithi115 ceph-mon[36174]: osdmap e110: 12 total, 12 up, 12 in 2024-03-26T18:16:10.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:09 smithi049 ceph-mon[32331]: pgmap v438: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.3 MiB/s wr, 451 op/s 2024-03-26T18:16:10.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:09 smithi049 ceph-mon[32331]: osdmap e110: 12 total, 12 up, 12 in 2024-03-26T18:16:12.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:11 smithi171 ceph-mon[36033]: pgmap v440: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.5 MiB/s wr, 477 op/s 2024-03-26T18:16:12.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:11 smithi115 ceph-mon[36174]: pgmap v440: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.5 MiB/s wr, 477 op/s 2024-03-26T18:16:12.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:11 smithi049 ceph-mon[32331]: pgmap v440: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.5 MiB/s wr, 477 op/s 2024-03-26T18:16:13.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:12 smithi115 ceph-mon[36174]: pgmap v441: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-03-26T18:16:13.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:12 smithi049 ceph-mon[32331]: pgmap v441: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-03-26T18:16:13.349 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:12 smithi171 ceph-mon[36033]: pgmap v441: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-03-26T18:16:15.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 830.0 s, 533.4 tps, lat 187.514 ms stddev 303.370 2024-03-26T18:16:15.599 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:15 smithi171 ceph-mon[36033]: pgmap v442: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-03-26T18:16:15.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:15 smithi115 ceph-mon[36174]: pgmap v442: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-03-26T18:16:15.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:15 smithi049 ceph-mon[32331]: pgmap v442: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-03-26T18:16:17.599 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:17 smithi171 ceph-mon[36033]: pgmap v443: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.7 MiB/s wr, 489 op/s 2024-03-26T18:16:17.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:17 smithi115 ceph-mon[36174]: pgmap v443: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.7 MiB/s wr, 489 op/s 2024-03-26T18:16:17.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:17 smithi049 ceph-mon[32331]: pgmap v443: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.7 MiB/s wr, 489 op/s 2024-03-26T18:16:18.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:18 smithi115 ceph-mon[36174]: osdmap e111: 12 total, 12 up, 12 in 2024-03-26T18:16:18.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:18 smithi049 ceph-mon[32331]: osdmap e111: 12 total, 12 up, 12 in 2024-03-26T18:16:19.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:18 smithi171 ceph-mon[36033]: osdmap e111: 12 total, 12 up, 12 in 2024-03-26T18:16:19.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:19 smithi115 ceph-mon[36174]: pgmap v445: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-03-26T18:16:19.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:19 smithi049 ceph-mon[32331]: pgmap v445: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-03-26T18:16:20.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:19 smithi171 ceph-mon[36033]: pgmap v445: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-03-26T18:16:20.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 835.0 s, 512.2 tps, lat 196.688 ms stddev 249.002 2024-03-26T18:16:21.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:21 smithi115 ceph-mon[36174]: pgmap v446: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-03-26T18:16:21.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:21 smithi049 ceph-mon[32331]: pgmap v446: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-03-26T18:16:22.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:21 smithi171 ceph-mon[36033]: pgmap v446: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-03-26T18:16:23.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:23 smithi115 ceph-mon[36174]: pgmap v447: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.8 MiB/s wr, 505 op/s 2024-03-26T18:16:23.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:23 smithi049 ceph-mon[32331]: pgmap v447: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.8 MiB/s wr, 505 op/s 2024-03-26T18:16:24.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:23 smithi171 ceph-mon[36033]: pgmap v447: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 4.8 MiB/s wr, 505 op/s 2024-03-26T18:16:25.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 840.0 s, 491.4 tps, lat 205.650 ms stddev 226.157 2024-03-26T18:16:25.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:25 smithi115 ceph-mon[36174]: pgmap v448: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.9 MiB/s wr, 519 op/s 2024-03-26T18:16:25.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:25 smithi049 ceph-mon[32331]: pgmap v448: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.9 MiB/s wr, 519 op/s 2024-03-26T18:16:26.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:25 smithi171 ceph-mon[36033]: pgmap v448: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.9 MiB/s wr, 519 op/s 2024-03-26T18:16:27.980 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:27 smithi115 ceph-mon[36174]: pgmap v449: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.9 MiB/s wr, 531 op/s 2024-03-26T18:16:27.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:27 smithi049 ceph-mon[32331]: pgmap v449: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.9 MiB/s wr, 531 op/s 2024-03-26T18:16:28.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:27 smithi171 ceph-mon[36033]: pgmap v449: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.9 MiB/s wr, 531 op/s 2024-03-26T18:16:29.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:29 smithi115 ceph-mon[36174]: pgmap v450: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.9 MiB/s wr, 537 op/s 2024-03-26T18:16:29.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:29 smithi049 ceph-mon[32331]: pgmap v450: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.9 MiB/s wr, 537 op/s 2024-03-26T18:16:30.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:29 smithi171 ceph-mon[36033]: pgmap v450: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.9 MiB/s wr, 537 op/s 2024-03-26T18:16:30.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 845.0 s, 448.8 tps, lat 222.761 ms stddev 237.373 2024-03-26T18:16:31.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:31 smithi049 ceph-mon[32331]: pgmap v451: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-03-26T18:16:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:31 smithi171 ceph-mon[36033]: pgmap v451: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-03-26T18:16:32.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:31 smithi115 ceph-mon[36174]: pgmap v451: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.5 MiB/s wr, 502 op/s 2024-03-26T18:16:33.995 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:33 smithi049 ceph-mon[32331]: pgmap v452: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 4.6 MiB/s wr, 496 op/s 2024-03-26T18:16:34.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:33 smithi171 ceph-mon[36033]: pgmap v452: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 4.6 MiB/s wr, 496 op/s 2024-03-26T18:16:34.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:33 smithi115 ceph-mon[36174]: pgmap v452: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 4.6 MiB/s wr, 496 op/s 2024-03-26T18:16:35.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 850.0 s, 501.6 tps, lat 200.667 ms stddev 270.021 2024-03-26T18:16:36.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:35 smithi171 ceph-mon[36033]: pgmap v453: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-03-26T18:16:36.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:35 smithi115 ceph-mon[36174]: pgmap v453: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-03-26T18:16:36.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:35 smithi049 ceph-mon[32331]: pgmap v453: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-03-26T18:16:38.046 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:37 smithi049 ceph-mon[32331]: pgmap v454: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.8 MiB/s wr, 544 op/s 2024-03-26T18:16:38.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:37 smithi171 ceph-mon[36033]: pgmap v454: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.8 MiB/s wr, 544 op/s 2024-03-26T18:16:38.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:37 smithi115 ceph-mon[36174]: pgmap v454: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.8 MiB/s wr, 544 op/s 2024-03-26T18:16:40.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:39 smithi171 ceph-mon[36033]: pgmap v455: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.1 MiB/s wr, 455 op/s 2024-03-26T18:16:40.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 855.0 s, 474.4 tps, lat 211.716 ms stddev 235.035 2024-03-26T18:16:40.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:39 smithi115 ceph-mon[36174]: pgmap v455: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.1 MiB/s wr, 455 op/s 2024-03-26T18:16:40.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:39 smithi049 ceph-mon[32331]: pgmap v455: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.1 MiB/s wr, 455 op/s 2024-03-26T18:16:42.031 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:41 smithi115 ceph-mon[36174]: pgmap v456: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-03-26T18:16:42.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:41 smithi171 ceph-mon[36033]: pgmap v456: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-03-26T18:16:42.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:41 smithi049 ceph-mon[32331]: pgmap v456: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-03-26T18:16:44.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:43 smithi171 ceph-mon[36033]: pgmap v457: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.4 MiB/s wr, 486 op/s 2024-03-26T18:16:44.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:43 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:16:44.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:43 smithi115 ceph-mon[36174]: pgmap v457: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.4 MiB/s wr, 486 op/s 2024-03-26T18:16:44.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:43 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:16:44.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:43 smithi049 ceph-mon[32331]: pgmap v457: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.4 MiB/s wr, 486 op/s 2024-03-26T18:16:44.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:43 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:16:45.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:44 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:16:45.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:44 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:16:45.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:44 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:16:45.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 860.0 s, 442.8 tps, lat 224.376 ms stddev 253.033 2024-03-26T18:16:45.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:44 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:16:45.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:44 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:16:45.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:44 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:16:45.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:44 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:16:45.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:44 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:16:45.245 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:44 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:16:46.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:45 smithi171 ceph-mon[36033]: pgmap v458: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 4.0 MiB/s wr, 438 op/s 2024-03-26T18:16:46.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:45 smithi115 ceph-mon[36174]: pgmap v458: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 4.0 MiB/s wr, 438 op/s 2024-03-26T18:16:46.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:45 smithi049 ceph-mon[32331]: pgmap v458: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 4.0 MiB/s wr, 438 op/s 2024-03-26T18:16:48.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:47 smithi171 ceph-mon[36033]: pgmap v459: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-03-26T18:16:48.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:47 smithi115 ceph-mon[36174]: pgmap v459: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-03-26T18:16:48.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:47 smithi049 ceph-mon[32331]: pgmap v459: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.2 MiB/s wr, 478 op/s 2024-03-26T18:16:50.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:49 smithi171 ceph-mon[36033]: pgmap v460: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.0 MiB/s wr, 439 op/s 2024-03-26T18:16:50.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 865.0 s, 450.6 tps, lat 220.730 ms stddev 251.201 2024-03-26T18:16:50.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:49 smithi115 ceph-mon[36174]: pgmap v460: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.0 MiB/s wr, 439 op/s 2024-03-26T18:16:50.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:49 smithi049 ceph-mon[32331]: pgmap v460: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.0 MiB/s wr, 439 op/s 2024-03-26T18:16:52.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:51 smithi171 ceph-mon[36033]: pgmap v461: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-03-26T18:16:52.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:51 smithi115 ceph-mon[36174]: pgmap v461: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-03-26T18:16:52.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:51 smithi049 ceph-mon[32331]: pgmap v461: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-03-26T18:16:54.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:53 smithi171 ceph-mon[36033]: pgmap v462: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-03-26T18:16:54.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:53 smithi115 ceph-mon[36174]: pgmap v462: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-03-26T18:16:54.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:53 smithi049 ceph-mon[32331]: pgmap v462: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-03-26T18:16:55.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 870.0 s, 474.2 tps, lat 207.621 ms stddev 237.386 2024-03-26T18:16:56.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:55 smithi171 ceph-mon[36033]: pgmap v463: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 3.9 MiB/s wr, 424 op/s 2024-03-26T18:16:56.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:55 smithi115 ceph-mon[36174]: pgmap v463: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 3.9 MiB/s wr, 424 op/s 2024-03-26T18:16:56.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:55 smithi049 ceph-mon[32331]: pgmap v463: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 3.9 MiB/s wr, 424 op/s 2024-03-26T18:16:58.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:57 smithi171 ceph-mon[36033]: pgmap v464: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.0 MiB/s wr, 462 op/s 2024-03-26T18:16:58.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:57 smithi115 ceph-mon[36174]: pgmap v464: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.0 MiB/s wr, 462 op/s 2024-03-26T18:16:58.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:57 smithi049 ceph-mon[32331]: pgmap v464: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.0 MiB/s wr, 462 op/s 2024-03-26T18:16:59.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:16:58 smithi115 ceph-mon[36174]: pgmap v465: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 4.3 MiB/s wr, 467 op/s 2024-03-26T18:16:59.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:16:58 smithi049 ceph-mon[32331]: pgmap v465: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 4.3 MiB/s wr, 467 op/s 2024-03-26T18:16:59.348 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:16:58 smithi171 ceph-mon[36033]: pgmap v465: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 4.3 MiB/s wr, 467 op/s 2024-03-26T18:17:00.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 875.0 s, 441.2 tps, lat 226.757 ms stddev 261.005 2024-03-26T18:17:01.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:01 smithi171 ceph-mon[36033]: pgmap v466: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 3.8 MiB/s wr, 432 op/s 2024-03-26T18:17:01.729 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:01 smithi115 ceph-mon[36174]: pgmap v466: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 3.8 MiB/s wr, 432 op/s 2024-03-26T18:17:01.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:01 smithi049 ceph-mon[32331]: pgmap v466: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 3.8 MiB/s wr, 432 op/s 2024-03-26T18:17:03.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:03 smithi171 ceph-mon[36033]: pgmap v467: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.0 MiB/s wr, 459 op/s 2024-03-26T18:17:03.729 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:03 smithi115 ceph-mon[36174]: pgmap v467: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.0 MiB/s wr, 459 op/s 2024-03-26T18:17:03.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:03 smithi049 ceph-mon[32331]: pgmap v467: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 4.0 MiB/s wr, 459 op/s 2024-03-26T18:17:05.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 880.0 s, 438.6 tps, lat 233.807 ms stddev 272.723 2024-03-26T18:17:05.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:05 smithi171 ceph-mon[36033]: pgmap v468: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.8 MiB/s wr, 447 op/s 2024-03-26T18:17:05.729 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:05 smithi115 ceph-mon[36174]: pgmap v468: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.8 MiB/s wr, 447 op/s 2024-03-26T18:17:05.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:05 smithi049 ceph-mon[32331]: pgmap v468: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.8 MiB/s wr, 447 op/s 2024-03-26T18:17:07.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:07 smithi171 ceph-mon[36033]: pgmap v469: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 4.0 MiB/s wr, 468 op/s 2024-03-26T18:17:07.729 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:07 smithi115 ceph-mon[36174]: pgmap v469: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 4.0 MiB/s wr, 468 op/s 2024-03-26T18:17:07.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:07 smithi049 ceph-mon[32331]: pgmap v469: 129 pgs: 129 active+clean; 3.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 4.0 MiB/s wr, 468 op/s 2024-03-26T18:17:09.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:09 smithi171 ceph-mon[36033]: pgmap v470: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-03-26T18:17:09.729 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:09 smithi115 ceph-mon[36174]: pgmap v470: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-03-26T18:17:09.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:09 smithi049 ceph-mon[32331]: pgmap v470: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-03-26T18:17:10.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 885.0 s, 453.2 tps, lat 197.254 ms stddev 221.984 2024-03-26T18:17:10.729 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:10 smithi115 ceph-mon[36174]: osdmap e112: 12 total, 12 up, 12 in 2024-03-26T18:17:10.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:10 smithi049 ceph-mon[32331]: osdmap e112: 12 total, 12 up, 12 in 2024-03-26T18:17:10.848 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:10 smithi171 ceph-mon[36033]: osdmap e112: 12 total, 12 up, 12 in 2024-03-26T18:17:11.729 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:11 smithi115 ceph-mon[36174]: pgmap v472: 129 pgs: 5 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 106 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-03-26T18:17:11.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:11 smithi049 ceph-mon[32331]: pgmap v472: 129 pgs: 5 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 106 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-03-26T18:17:11.848 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:11 smithi171 ceph-mon[36033]: pgmap v472: 129 pgs: 5 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 106 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-03-26T18:17:13.729 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:13 smithi115 ceph-mon[36174]: pgmap v473: 129 pgs: 7 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 99 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 4.0 MiB/s wr, 474 op/s 2024-03-26T18:17:13.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:13 smithi049 ceph-mon[32331]: pgmap v473: 129 pgs: 7 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 99 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 4.0 MiB/s wr, 474 op/s 2024-03-26T18:17:13.848 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:13 smithi171 ceph-mon[36033]: pgmap v473: 129 pgs: 7 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 99 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 4.0 MiB/s wr, 474 op/s 2024-03-26T18:17:15.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 890.0 s, 431.0 tps, lat 249.095 ms stddev 323.795 2024-03-26T18:17:15.729 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:15 smithi115 ceph-mon[36174]: pgmap v474: 129 pgs: 7 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 99 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 3.9 MiB/s wr, 456 op/s 2024-03-26T18:17:15.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:15 smithi049 ceph-mon[32331]: pgmap v474: 129 pgs: 7 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 99 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 3.9 MiB/s wr, 456 op/s 2024-03-26T18:17:15.848 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:15 smithi171 ceph-mon[36033]: pgmap v474: 129 pgs: 7 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 99 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 3.9 MiB/s wr, 456 op/s 2024-03-26T18:17:17.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:17 smithi115 ceph-mon[36174]: pgmap v475: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 3.9 MiB/s wr, 462 op/s 2024-03-26T18:17:17.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:17 smithi049 ceph-mon[32331]: pgmap v475: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 3.9 MiB/s wr, 462 op/s 2024-03-26T18:17:17.848 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:17 smithi171 ceph-mon[36033]: pgmap v475: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 3.9 MiB/s wr, 462 op/s 2024-03-26T18:17:18.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:18 smithi115 ceph-mon[36174]: osdmap e113: 12 total, 12 up, 12 in 2024-03-26T18:17:18.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:18 smithi049 ceph-mon[32331]: osdmap e113: 12 total, 12 up, 12 in 2024-03-26T18:17:19.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:18 smithi171 ceph-mon[36033]: osdmap e113: 12 total, 12 up, 12 in 2024-03-26T18:17:19.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:19 smithi115 ceph-mon[36174]: pgmap v477: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-03-26T18:17:19.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:19 smithi049 ceph-mon[32331]: pgmap v477: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-03-26T18:17:20.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:19 smithi171 ceph-mon[36033]: pgmap v477: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-03-26T18:17:20.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 895.0 s, 475.8 tps, lat 217.247 ms stddev 266.581 2024-03-26T18:17:21.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:21 smithi115 ceph-mon[36174]: pgmap v478: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 3.9 MiB/s wr, 458 op/s 2024-03-26T18:17:21.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:21 smithi049 ceph-mon[32331]: pgmap v478: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 3.9 MiB/s wr, 458 op/s 2024-03-26T18:17:22.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:21 smithi171 ceph-mon[36033]: pgmap v478: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 3.9 MiB/s wr, 458 op/s 2024-03-26T18:17:23.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:23 smithi115 ceph-mon[36174]: pgmap v479: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.3 MiB/s wr, 477 op/s 2024-03-26T18:17:23.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:23 smithi049 ceph-mon[32331]: pgmap v479: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.3 MiB/s wr, 477 op/s 2024-03-26T18:17:24.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:23 smithi171 ceph-mon[36033]: pgmap v479: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.3 MiB/s wr, 477 op/s 2024-03-26T18:17:25.202 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 900.0 s, 372.8 tps, lat 256.354 ms stddev 303.886 2024-03-26T18:17:25.585 INFO:teuthology.orchestra.run.smithi049.stdout:transaction type: 2024-03-26T18:17:25.586 INFO:teuthology.orchestra.run.smithi049.stdout:scaling factor: 32 2024-03-26T18:17:25.586 INFO:teuthology.orchestra.run.smithi049.stdout:query mode: simple 2024-03-26T18:17:25.586 INFO:teuthology.orchestra.run.smithi049.stdout:number of clients: 100 2024-03-26T18:17:25.586 INFO:teuthology.orchestra.run.smithi049.stdout:number of threads: 4 2024-03-26T18:17:25.586 INFO:teuthology.orchestra.run.smithi049.stdout:duration: 900 s 2024-03-26T18:17:25.586 INFO:teuthology.orchestra.run.smithi049.stdout:number of transactions actually processed: 805668 2024-03-26T18:17:25.586 INFO:teuthology.orchestra.run.smithi049.stdout:latency average = 111.714 ms 2024-03-26T18:17:25.586 INFO:teuthology.orchestra.run.smithi049.stdout:latency stddev = 181.685 ms 2024-03-26T18:17:25.586 INFO:teuthology.orchestra.run.smithi049.stdout:tps = 894.805413 (including connections establishing) 2024-03-26T18:17:25.586 INFO:teuthology.orchestra.run.smithi049.stdout:tps = 894.807675 (excluding connections establishing) 2024-03-26T18:17:25.617 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-03-26T18:17:25.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:25 smithi115 ceph-mon[36174]: pgmap v480: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 4.4 MiB/s wr, 463 op/s 2024-03-26T18:17:25.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:25 smithi049 ceph-mon[32331]: pgmap v480: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 4.4 MiB/s wr, 463 op/s 2024-03-26T18:17:26.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:25 smithi171 ceph-mon[36033]: pgmap v480: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 4.4 MiB/s wr, 463 op/s 2024-03-26T18:17:27.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:27 smithi115 ceph-mon[36174]: pgmap v481: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 5.3 MiB/s wr, 497 op/s 2024-03-26T18:17:27.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:27 smithi049 ceph-mon[32331]: pgmap v481: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 5.3 MiB/s wr, 497 op/s 2024-03-26T18:17:28.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:27 smithi171 ceph-mon[36033]: pgmap v481: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 5.3 MiB/s wr, 497 op/s 2024-03-26T18:17:29.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:29 smithi049 ceph-mon[32331]: pgmap v482: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 5.4 MiB/s wr, 464 op/s 2024-03-26T18:17:30.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:29 smithi171 ceph-mon[36033]: pgmap v482: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 5.4 MiB/s wr, 464 op/s 2024-03-26T18:17:30.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:29 smithi115 ceph-mon[36174]: pgmap v482: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 5.4 MiB/s wr, 464 op/s 2024-03-26T18:17:31.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:31 smithi049 ceph-mon[32331]: pgmap v483: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 5.7 MiB/s wr, 515 op/s 2024-03-26T18:17:32.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:31 smithi171 ceph-mon[36033]: pgmap v483: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 5.7 MiB/s wr, 515 op/s 2024-03-26T18:17:32.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:31 smithi115 ceph-mon[36174]: pgmap v483: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 5.7 MiB/s wr, 515 op/s 2024-03-26T18:17:34.101 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:33 smithi171 ceph-mon[36033]: pgmap v484: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 6.3 MiB/s wr, 570 op/s 2024-03-26T18:17:34.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:33 smithi115 ceph-mon[36174]: pgmap v484: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 6.3 MiB/s wr, 570 op/s 2024-03-26T18:17:34.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:33 smithi049 ceph-mon[32331]: pgmap v484: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 6.3 MiB/s wr, 570 op/s 2024-03-26T18:17:36.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:35 smithi171 ceph-mon[36033]: pgmap v485: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 6.7 MiB/s wr, 628 op/s 2024-03-26T18:17:36.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:35 smithi115 ceph-mon[36174]: pgmap v485: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 6.7 MiB/s wr, 628 op/s 2024-03-26T18:17:36.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:35 smithi049 ceph-mon[32331]: pgmap v485: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 6.7 MiB/s wr, 628 op/s 2024-03-26T18:17:38.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:37 smithi171 ceph-mon[36033]: pgmap v486: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 7.0 MiB/s wr, 674 op/s 2024-03-26T18:17:38.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:37 smithi115 ceph-mon[36174]: pgmap v486: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 7.0 MiB/s wr, 674 op/s 2024-03-26T18:17:38.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:37 smithi049 ceph-mon[32331]: pgmap v486: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 7.0 MiB/s wr, 674 op/s 2024-03-26T18:17:39.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-26T18:17:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/: 2024-03-26T18:17:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:17:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 .. 2024-03-26T18:17:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 data 2024-03-26T18:17:39.569 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data: 2024-03-26T18:17:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:total 36 2024-03-26T18:17:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 . 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 .. 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:02 base 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Mar 26 18:02 current_logfiles 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Mar 26 18:02 global 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 log 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_commit_ts 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_dynshmem 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Mar 26 18:02 pg_hba.conf 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Mar 26 18:02 pg_ident.conf 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:17 pg_logical 2024-03-26T18:17:39.570 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 pg_multixact 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_notify 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_replslot 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_serial 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_snapshots 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:17 pg_stat 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:17 pg_stat_tmp 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:17 pg_subtrans 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_tblspc 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pg_twophase 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 PG_VERSION 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 60 Mar 26 18:17 pg_wal 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 pg_xact 2024-03-26T18:17:39.571 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Mar 26 18:02 postgresql.auto.conf 2024-03-26T18:17:39.572 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Mar 26 18:02 postgresql.conf 2024-03-26T18:17:39.572 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Mar 26 18:02 postmaster.opts 2024-03-26T18:17:39.572 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.572 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base: 2024-03-26T18:17:39.572 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.572 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:02 . 2024-03-26T18:17:39.572 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.572 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:02 1 2024-03-26T18:17:39.572 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:02 13493 2024-03-26T18:17:39.572 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Mar 26 18:08 13494 2024-03-26T18:17:39.573 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 pgsql_tmp 2024-03-26T18:17:39.573 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.573 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/1: 2024-03-26T18:17:39.573 INFO:teuthology.orchestra.run.smithi049.stdout:total 7777 2024-03-26T18:17:39.573 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:02 . 2024-03-26T18:17:39.573 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:02 .. 2024-03-26T18:17:39.573 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 112 2024-03-26T18:17:39.573 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 113 2024-03-26T18:17:39.573 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 1247 2024-03-26T18:17:39.573 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1247_fsm 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1247_vm 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:02 1249 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1249_fsm 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1249_vm 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:02 1255 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1255_fsm 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1255_vm 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:02 1259 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1259_fsm 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1259_vm 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:02 13344 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13344_fsm 2024-03-26T18:17:39.574 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13344_vm 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13346 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13348 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13349_fsm 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349_vm 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13351 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13353 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13354_fsm 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354_vm 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13356 2024-03-26T18:17:39.575 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13358 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13359_fsm 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359_vm 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13361 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13363 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1417 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1418 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 174 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 175 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2187 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2224 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2328 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2336 2024-03-26T18:17:39.576 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2337 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2579 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2600 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2600_fsm 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2600_vm 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2601_fsm 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601_vm 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2602 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2602_fsm 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2602_vm 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2603 2024-03-26T18:17:39.577 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2603_fsm 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2603_vm 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2604 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2605 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2605_fsm 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2605_vm 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2606_fsm 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606_vm 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2607 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2607_fsm 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2607_vm 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:02 2608 2024-03-26T18:17:39.578 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2608_fsm 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2608_vm 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:02 2609 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2609_fsm 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2609_vm 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2610 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2610_fsm 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2610_vm 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2611 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2612_fsm 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612_vm 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2613 2024-03-26T18:17:39.579 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2615_fsm 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615_vm 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2616 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2616_fsm 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2616_vm 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:02 2617 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2617_fsm 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2617_vm 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:02 2618 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2618_fsm 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2618_vm 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 18:02 2619 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2619_fsm 2024-03-26T18:17:39.580 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2619_vm 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2620 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2650 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2651 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2652 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2653 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2654 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2655 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2656 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2657 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:02 2658 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 2659 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2660 2024-03-26T18:17:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2661 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2662 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2663 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2664 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2665 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2666 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2667 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2668 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2669 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2670 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:02 2673 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:02 2674 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:02 2675 2024-03-26T18:17:39.582 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2678 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2679 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2680 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2681 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2682 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2683 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2684 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2685 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2686 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2687 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2688 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2689 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:02 2690 2024-03-26T18:17:39.583 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:02 2691 2024-03-26T18:17:39.584 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2692 2024-03-26T18:17:39.584 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2693 2024-03-26T18:17:39.584 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2696 2024-03-26T18:17:39.584 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2699 2024-03-26T18:17:39.584 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2701 2024-03-26T18:17:39.584 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2702 2024-03-26T18:17:39.584 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2703 2024-03-26T18:17:39.584 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2704 2024-03-26T18:17:39.584 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2753 2024-03-26T18:17:39.584 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2753_fsm 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2753_vm 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2754 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2755 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2756 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2757 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2830 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2831 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2832 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2833 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2834 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2835 2024-03-26T18:17:39.585 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2836 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2837 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:02 2838 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2838_fsm 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2838_vm 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2839 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840_fsm 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2840_vm 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2841 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2995 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2996 2024-03-26T18:17:39.586 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3079_fsm 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079_vm 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3080 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3081 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3085 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3118 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3119 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:02 3164 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3256 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3257 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3258 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3350 2024-03-26T18:17:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3351 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3379 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3380 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3381 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3394 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3394_fsm 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3394_vm 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3395 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3429 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3430 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3431 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3433 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3439 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3440 2024-03-26T18:17:39.588 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3455 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:02 3456 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3456_fsm 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3456_vm 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3466 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3467 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3468 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3501 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3502 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3503 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3534 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3541_fsm 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541_vm 2024-03-26T18:17:39.589 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3542 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3574 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3575 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3576 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3596 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3597 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3598 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3599 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3600_fsm 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600_vm 2024-03-26T18:17:39.590 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3601_fsm 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601_vm 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3602_fsm 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602_vm 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603_fsm 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3603_vm 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3604 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3605 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3606 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3607 2024-03-26T18:17:39.591 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3608 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 3609 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3712 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3764_fsm 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764_vm 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3766 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3767 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3997 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4143 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4144 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4145 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4146 2024-03-26T18:17:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4147 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4148 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4149 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4150 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4151 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4152 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4153 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4154 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4155 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4156 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4157 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4158 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4159 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4160 2024-03-26T18:17:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4161 2024-03-26T18:17:39.594 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4162 2024-03-26T18:17:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4163 2024-03-26T18:17:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4164 2024-03-26T18:17:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4165 2024-03-26T18:17:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4166 2024-03-26T18:17:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4167 2024-03-26T18:17:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4168 2024-03-26T18:17:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4169 2024-03-26T18:17:39.595 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4170 2024-03-26T18:17:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4171 2024-03-26T18:17:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4172 2024-03-26T18:17:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4173 2024-03-26T18:17:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4174 2024-03-26T18:17:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 5002 2024-03-26T18:17:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 548 2024-03-26T18:17:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 549 2024-03-26T18:17:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6102 2024-03-26T18:17:39.596 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6104 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6106 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6110 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6111 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6112 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6113 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6117 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 826 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 827 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 828 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:02 pg_filenode.map 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 PG_VERSION 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-03-26T18:17:39.597 INFO:teuthology.orchestra.run.smithi049.stdout:total 7777 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 18:02 . 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:02 .. 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 112 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 113 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 1247 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1247_fsm 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1247_vm 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:02 1249 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1249_fsm 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1249_vm 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:02 1255 2024-03-26T18:17:39.598 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1255_fsm 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1255_vm 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:02 1259 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1259_fsm 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1259_vm 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:02 13344 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13344_fsm 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13344_vm 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13346 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13348 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13349_fsm 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349_vm 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13351 2024-03-26T18:17:39.599 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13353 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13354_fsm 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354_vm 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13356 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13358 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13359_fsm 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359_vm 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13361 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13363 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1417 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1418 2024-03-26T18:17:39.600 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 174 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 175 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2187 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2224 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2328 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2336 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2337 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2579 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2600 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2600_fsm 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2600_vm 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2601_fsm 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601_vm 2024-03-26T18:17:39.601 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2602 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2602_fsm 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2602_vm 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2603 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2603_fsm 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2603_vm 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2604 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2605 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2605_fsm 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2605_vm 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2606_fsm 2024-03-26T18:17:39.602 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606_vm 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2607 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2607_fsm 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2607_vm 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:02 2608 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2608_fsm 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2608_vm 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:02 2609 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2609_fsm 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2609_vm 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2610 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2610_fsm 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2610_vm 2024-03-26T18:17:39.603 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2611 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2612_fsm 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612_vm 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2613 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2615_fsm 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615_vm 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2616 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2616_fsm 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2616_vm 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:02 2617 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2617_fsm 2024-03-26T18:17:39.604 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2617_vm 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:02 2618 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2618_fsm 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2618_vm 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 18:02 2619 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2619_fsm 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2619_vm 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2620 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2650 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2651 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2652 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2653 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2654 2024-03-26T18:17:39.605 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2655 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2656 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2657 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:02 2658 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 2659 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2660 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2661 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2662 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2663 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2664 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2665 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2666 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2667 2024-03-26T18:17:39.606 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2668 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2669 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2670 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:02 2673 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:02 2674 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:02 2675 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2678 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2679 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2680 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2681 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2682 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2683 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2684 2024-03-26T18:17:39.607 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2685 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2686 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2687 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2688 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2689 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:02 2690 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:02 2691 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2692 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2693 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2696 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2699 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2701 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2702 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2703 2024-03-26T18:17:39.608 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2704 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2753 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2753_fsm 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2753_vm 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2754 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2755 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2756 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2757 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2830 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2831 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2832 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2833 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2834 2024-03-26T18:17:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2835 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2836 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2837 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:02 2838 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2838_fsm 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2838_vm 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2839 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840_fsm 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2840_vm 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2841 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2995 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2996 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079 2024-03-26T18:17:39.610 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3079_fsm 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079_vm 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3080 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3081 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3085 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3118 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3119 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:02 3164 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3256 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3257 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3258 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3350 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3351 2024-03-26T18:17:39.611 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3379 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3380 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3381 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3394 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3394_fsm 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3394_vm 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3395 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3429 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3430 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3431 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3433 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3439 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3440 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3455 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:02 3456 2024-03-26T18:17:39.612 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3456_fsm 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3456_vm 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3466 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3467 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3468 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3501 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3502 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3503 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3534 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3541_fsm 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541_vm 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3542 2024-03-26T18:17:39.613 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3574 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3575 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3576 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3596 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3597 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3598 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3599 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3600_fsm 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600_vm 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3601_fsm 2024-03-26T18:17:39.614 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601_vm 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3602_fsm 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602_vm 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603_fsm 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3603_vm 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3604 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3605 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3606 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3607 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3608 2024-03-26T18:17:39.615 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 3609 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3712 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3764_fsm 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764_vm 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3766 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3767 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3997 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4143 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4144 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4145 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4146 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4147 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4148 2024-03-26T18:17:39.616 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4149 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4150 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4151 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4152 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4153 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4154 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4155 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4156 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4157 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4158 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4159 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4160 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4161 2024-03-26T18:17:39.617 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4162 2024-03-26T18:17:39.618 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4163 2024-03-26T18:17:39.618 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4164 2024-03-26T18:17:39.618 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4165 2024-03-26T18:17:39.618 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4166 2024-03-26T18:17:39.618 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4167 2024-03-26T18:17:39.618 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4168 2024-03-26T18:17:39.618 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4169 2024-03-26T18:17:39.618 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4170 2024-03-26T18:17:39.618 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4171 2024-03-26T18:17:39.618 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4172 2024-03-26T18:17:39.620 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4173 2024-03-26T18:17:39.620 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4174 2024-03-26T18:17:39.620 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 5002 2024-03-26T18:17:39.620 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 548 2024-03-26T18:17:39.620 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 549 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6102 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6104 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6106 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6110 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6111 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6112 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6113 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6117 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 826 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 827 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 828 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:02 pg_filenode.map 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 PG_VERSION 2024-03-26T18:17:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:total 551350 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Mar 26 18:08 . 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:02 .. 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 112 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 113 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 1247 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1247_fsm 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1247_vm 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 18:02 1249 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1249_fsm 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1249_vm 2024-03-26T18:17:39.622 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 18:02 1255 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1255_fsm 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1255_vm 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 18:17 1259 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1259_fsm 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1259_vm 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 18:02 13344 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13344_fsm 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13344_vm 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13346 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13348 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13349_fsm 2024-03-26T18:17:39.623 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13349_vm 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13351 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13353 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13354_fsm 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13354_vm 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13356 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13358 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 13359_fsm 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13359_vm 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 13361 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 13363 2024-03-26T18:17:39.624 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1417 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 1418 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 440090624 Mar 26 18:17 16396 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Mar 26 18:17 16396_fsm 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:14 16396_vm 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Mar 26 18:17 16397 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:17 16397_fsm 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:17 16397_vm 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1097728 Mar 26 18:17 16399 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:17 16399_fsm 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:17 16399_vm 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:17 16400 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:17 16402 2024-03-26T18:17:39.625 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Mar 26 18:17 16404 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 42631168 Mar 26 18:17 16406 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:17 16406_fsm 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:17 16406_vm 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 174 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 175 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2187 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2224 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2328 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2336 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2337 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2579 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2600 2024-03-26T18:17:39.626 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2600_fsm 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2600_vm 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2601_fsm 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2601_vm 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2602 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2602_fsm 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2602_vm 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2603 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2603_fsm 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2603_vm 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2604 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2605 2024-03-26T18:17:39.627 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2605_fsm 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2605_vm 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2606_fsm 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2606_vm 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2607 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2607_fsm 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2607_vm 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 18:02 2608 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2608_fsm 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2608_vm 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 18:02 2609 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2609_fsm 2024-03-26T18:17:39.628 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2609_vm 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2610 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2610_fsm 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2610_vm 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2611 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2612_fsm 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2612_vm 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2613 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2615_fsm 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2615_vm 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2616 2024-03-26T18:17:39.629 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2616_fsm 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2616_vm 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 18:02 2617 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2617_fsm 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2617_vm 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 18:02 2618 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2618_fsm 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2618_vm 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Mar 26 18:17 2619 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:05 2619_fsm 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2619_vm 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2620 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2650 2024-03-26T18:17:39.630 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2651 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2652 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2653 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2654 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2655 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2656 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2657 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 18:02 2658 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:02 2659 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2660 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2661 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2662 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2663 2024-03-26T18:17:39.631 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2664 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2665 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2666 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2667 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2668 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2669 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2670 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 18:02 2673 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 18:02 2674 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 18:02 2675 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2678 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2679 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2680 2024-03-26T18:17:39.632 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2681 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2682 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2683 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2684 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2685 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2686 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2687 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2688 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 18:02 2689 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 18:02 2690 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 18:02 2691 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2692 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2693 2024-03-26T18:17:39.633 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:06 2696 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2699 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2701 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2702 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2703 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 2704 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2753 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2753_fsm 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2753_vm 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2754 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2755 2024-03-26T18:17:39.634 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2756 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 2757 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2830 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2831 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2832 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2833 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2834 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2835 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2836 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2837 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 18:02 2838 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2838_fsm 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2838_vm 2024-03-26T18:17:39.635 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2839 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2840_fsm 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2840_vm 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2841 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2995 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2996 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3079_fsm 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3079_vm 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3080 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3081 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3085 2024-03-26T18:17:39.636 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3118 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3119 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 18:02 3164 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3256 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3257 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3258 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3350 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3351 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3379 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3380 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3381 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3394 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3394_fsm 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3394_vm 2024-03-26T18:17:39.637 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3395 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3429 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3430 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3431 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3433 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3439 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3440 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 18:02 3455 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 18:02 3456 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3456_fsm 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3456_vm 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3466 2024-03-26T18:17:39.638 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3467 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3468 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3501 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3502 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3503 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3534 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3541_fsm 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3541_vm 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3542 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3574 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3575 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3576 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3596 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3597 2024-03-26T18:17:39.639 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3598 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3599 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3600_fsm 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3600_vm 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3601_fsm 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3601_vm 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3602_fsm 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3602_vm 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3603_fsm 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3603_vm 2024-03-26T18:17:39.640 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3604 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3605 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3606 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3607 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3608 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 18:02 3609 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3712 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 3764_fsm 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3764_vm 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3766 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 3767 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3997 2024-03-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4143 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4144 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4145 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4146 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4147 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4148 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4149 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4150 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4151 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4152 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4153 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4154 2024-03-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4155 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4156 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4157 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4158 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4159 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4160 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4161 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4162 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4163 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4164 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4165 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4166 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4167 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4168 2024-03-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4169 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4170 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4171 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4172 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4173 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4174 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 5002 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 548 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 549 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6102 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6104 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6106 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6110 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6111 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6112 2024-03-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6113 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6117 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 826 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 827 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 828 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:02 pg_filenode.map 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Mar 26 18:02 pg_internal.init 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:02 PG_VERSION 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 18:02 .. 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/global: 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:total 560 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Mar 26 18:02 . 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1213 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1213_fsm 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1213_vm 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1214 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1214_fsm 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1214_vm 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 1232 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 1233 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1260 2024-03-26T18:17:39.646 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1260_fsm 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1260_vm 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1261 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1261_fsm 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1261_vm 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1262 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 1262_fsm 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 1262_vm 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2396 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 18:02 2396_fsm 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2396_vm 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2397 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2671 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2672 2024-03-26T18:17:39.647 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2676 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2677 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2694 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2695 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2697 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 18:02 2698 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2846 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2847 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2964 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2965 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 2966 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 2967 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 3592 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 3593 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4060 2024-03-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4061 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4175 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4176 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4177 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4178 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4181 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4182 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4183 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4184 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 4185 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 4186 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6000 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6001 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6002 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 6100 2024-03-26T18:17:39.649 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6114 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 6115 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:17 pg_control 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 18:02 pg_filenode.map 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Mar 26 18:02 pg_internal.init 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/log: 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout:total 2 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1331 Mar 26 18:17 postgresql-Tue.log 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.650 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:total 1 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:17 . 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.651 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 mappings 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Mar 26 18:17 replorigin_checkpoint 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 snapshots 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:17 .. 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:17 .. 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.652 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 . 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 members 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 offsets 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 .. 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:02 0000 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.653 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 18:02 .. 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 18:07 0000 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.654 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.655 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:total 15 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 18:17 . 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Mar 26 18:17 db_0.stat 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Mar 26 18:17 db_13494.stat 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Mar 26 18:17 global.stat 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:17 . 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.656 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout:total 80 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:17 . 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 18:17 000C 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-03-26T18:17:39.657 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:total 966656 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 60 Mar 26 18:17 . 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:17 000000010000000000000064 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 000000010000000000000065 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:09 000000010000000000000066 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 000000010000000000000067 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 000000010000000000000068 2024-03-26T18:17:39.658 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:09 000000010000000000000069 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 00000001000000000000006A 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:08 00000001000000000000006B 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:08 00000001000000000000006C 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 00000001000000000000006D 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:10 00000001000000000000006E 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:10 00000001000000000000006F 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:08 000000010000000000000070 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:08 000000010000000000000071 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:10 000000010000000000000072 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:08 000000010000000000000073 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:09 000000010000000000000074 2024-03-26T18:17:39.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 000000010000000000000075 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 000000010000000000000076 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:08 000000010000000000000077 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 000000010000000000000078 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:11 000000010000000000000079 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 00000001000000000000007A 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:08 00000001000000000000007B 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:08 00000001000000000000007C 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 00000001000000000000007D 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:09 00000001000000000000007E 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:08 00000001000000000000007F 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 000000010000000000000080 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:07 000000010000000000000081 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:12 000000010000000000000082 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:13 000000010000000000000083 2024-03-26T18:17:39.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:13 000000010000000000000084 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:14 000000010000000000000085 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:14 000000010000000000000086 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:17 000000010000000000000087 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:12 000000010000000000000088 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:13 000000010000000000000089 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:16 00000001000000000000008A 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:15 00000001000000000000008B 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:13 00000001000000000000008C 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:15 00000001000000000000008D 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:13 00000001000000000000008E 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:14 00000001000000000000008F 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:12 000000010000000000000090 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:14 000000010000000000000091 2024-03-26T18:17:39.661 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:12 000000010000000000000092 2024-03-26T18:17:39.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:14 000000010000000000000093 2024-03-26T18:17:39.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:15 000000010000000000000094 2024-03-26T18:17:39.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:12 000000010000000000000095 2024-03-26T18:17:39.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:12 000000010000000000000096 2024-03-26T18:17:39.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:13 000000010000000000000097 2024-03-26T18:17:39.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:13 000000010000000000000098 2024-03-26T18:17:39.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:16 000000010000000000000099 2024-03-26T18:17:39.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:13 00000001000000000000009A 2024-03-26T18:17:39.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:12 00000001000000000000009B 2024-03-26T18:17:39.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:12 00000001000000000000009C 2024-03-26T18:17:39.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:12 00000001000000000000009D 2024-03-26T18:17:39.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 18:15 00000001000000000000009E 2024-03-26T18:17:39.663 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 archive_status 2024-03-26T18:17:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.665 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-03-26T18:17:39.665 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2024-03-26T18:17:39.665 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 18:02 . 2024-03-26T18:17:39.665 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 60 Mar 26 18:17 .. 2024-03-26T18:17:39.665 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:39.666 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-03-26T18:17:39.666 INFO:teuthology.orchestra.run.smithi049.stdout:total 200 2024-03-26T18:17:39.666 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 18:02 . 2024-03-26T18:17:39.666 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 18:17 .. 2024-03-26T18:17:39.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 204800 Mar 26 18:17 0000 2024-03-26T18:17:39.666 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-26T18:17:39.677 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-26T18:17:39.677 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T18:17:39.678 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T18:17:39.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-26T18:17:39.730 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-26T18:17:39.773 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-26T18:17:39.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:39 smithi049 ceph-mon[32331]: pgmap v487: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 5.7 MiB/s wr, 563 op/s 2024-03-26T18:17:40.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:39 smithi171 ceph-mon[36033]: pgmap v487: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 5.7 MiB/s wr, 563 op/s 2024-03-26T18:17:40.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:39 smithi115 ceph-mon[36174]: pgmap v487: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 5.7 MiB/s wr, 563 op/s 2024-03-26T18:17:40.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.704+0000 7f99f587e640 1 -- 172.21.15.49:0/1334582775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0109ab0 msgr2=0x7f99f0109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:40.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.704+0000 7f99f587e640 1 --2- 172.21.15.49:0/1334582775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0109ab0 0x7f99f0109e90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009c10 tx=0x7f99e802bef0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.704+0000 7f99f587e640 1 -- 172.21.15.49:0/1334582775 shutdown_connections 2024-03-26T18:17:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.704+0000 7f99f587e640 1 --2- 172.21.15.49:0/1334582775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99f0103aa0 0x7f99f0103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.704+0000 7f99f587e640 1 --2- 172.21.15.49:0/1334582775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99f0104460 0x7f99f0110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.704+0000 7f99f587e640 1 --2- 172.21.15.49:0/1334582775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0109ab0 0x7f99f0109e90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.704+0000 7f99f587e640 1 -- 172.21.15.49:0/1334582775 >> 172.21.15.49:0/1334582775 conn(0x7f99f00ff570 msgr2=0x7f99f0101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.704+0000 7f99f587e640 1 -- 172.21.15.49:0/1334582775 shutdown_connections 2024-03-26T18:17:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.704+0000 7f99f587e640 1 -- 172.21.15.49:0/1334582775 wait complete. 2024-03-26T18:17:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.705+0000 7f99f587e640 1 Processor -- start 2024-03-26T18:17:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99f587e640 1 -- start start 2024-03-26T18:17:40.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99f587e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99f0103aa0 0x7f99f019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99f587e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0104460 0x7f99f019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99f587e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99f0109ab0 0x7f99f019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99f587e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0113ec0 con 0x7f99f0104460 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99f587e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0113d40 con 0x7f99f0103aa0 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99f587e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0114040 con 0x7f99f0109ab0 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99ee7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0104460 0x7f99f019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99ee7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0104460 0x7f99f019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35120/0 (socket says 172.21.15.49:35120) 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99ee7fc640 1 -- 172.21.15.49:0/785413593 learned_addr learned my addr 172.21.15.49:0/785413593 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99eeffd640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99f0103aa0 0x7f99f019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99ef7fe640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99f0109ab0 0x7f99f019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99eeffd640 1 -- 172.21.15.49:0/785413593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99f0109ab0 msgr2=0x7f99f019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99eeffd640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99f0109ab0 0x7f99f019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99eeffd640 1 -- 172.21.15.49:0/785413593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0104460 msgr2=0x7f99f019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99eeffd640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0104460 0x7f99f019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99eeffd640 1 -- 172.21.15.49:0/785413593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f019f340 con 0x7f99f0103aa0 2024-03-26T18:17:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99ef7fe640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99f0109ab0 0x7f99f019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:17:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.706+0000 7f99ee7fc640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0104460 0x7f99f019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:17:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.707+0000 7f99eeffd640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99f0103aa0 0x7f99f019a3a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009be0 tx=0x7f99e80336b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.707+0000 7f99f487c640 1 -- 172.21.15.49:0/785413593 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8043020 con 0x7f99f0103aa0 2024-03-26T18:17:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.707+0000 7f99f487c640 1 -- 172.21.15.49:0/785413593 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e802ca80 con 0x7f99f0103aa0 2024-03-26T18:17:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.707+0000 7f99f487c640 1 -- 172.21.15.49:0/785413593 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e803e6b0 con 0x7f99f0103aa0 2024-03-26T18:17:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.707+0000 7f99f587e640 1 -- 172.21.15.49:0/785413593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f019f5d0 con 0x7f99f0103aa0 2024-03-26T18:17:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.709+0000 7f99f487c640 1 -- 172.21.15.49:0/785413593 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f99e8046020 con 0x7f99f0103aa0 2024-03-26T18:17:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.710+0000 7f99f487c640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f99dc0a3290 0x7f99dc0a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.710+0000 7f99ee7fc640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f99dc0a3290 0x7f99dc0a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.710+0000 7f99ee7fc640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f99dc0a3290 0x7f99dc0a5750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99f0101bd0 tx=0x7f99d8005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:40.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.713+0000 7f99f587e640 1 -- 172.21.15.49:0/785413593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f01a6ec0 con 0x7f99f0103aa0 2024-03-26T18:17:40.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.714+0000 7f99f487c640 1 -- 172.21.15.49:0/785413593 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f99e80338a0 con 0x7f99f0103aa0 2024-03-26T18:17:40.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.714+0000 7f99f587e640 1 -- 172.21.15.49:0/785413593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f01051e0 con 0x7f99f0103aa0 2024-03-26T18:17:40.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.743+0000 7f99f487c640 1 -- 172.21.15.49:0/785413593 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f99e80bdd00 con 0x7f99f0103aa0 2024-03-26T18:17:40.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.949+0000 7f99f587e640 1 -- 172.21.15.49:0/785413593 --> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f99f019b6f0 con 0x7f99dc0a3290 2024-03-26T18:17:40.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.956+0000 7f99f487c640 1 -- 172.21.15.49:0/785413593 <== mgr.15516 v2:172.21.15.49:6800/2879593470 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f99d8005020 con 0x7f99dc0a3290 2024-03-26T18:17:40.931 INFO:teuthology.orchestra.run.smithi049.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-26T00:00:00", "created": "2024-03-26T18:02:07", "first": "2024-03-26T18:03:00", "last": "2024-03-26T18:17:01", "last_pruned": "2024-03-26T18:17:01", "created_count": 15, "pruned_count": 9, "active": true} 2024-03-26T18:17:40.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 -- 172.21.15.49:0/785413593 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f99dc0a3290 msgr2=0x7f99dc0a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:40.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f99dc0a3290 0x7f99dc0a5750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99f0101bd0 tx=0x7f99d8005f90 comp rx=0 tx=0).stop 2024-03-26T18:17:40.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 -- 172.21.15.49:0/785413593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99f0103aa0 msgr2=0x7f99f019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:40.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99f0103aa0 0x7f99f019a3a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009be0 tx=0x7f99e80336b0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 -- 172.21.15.49:0/785413593 shutdown_connections 2024-03-26T18:17:40.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f99dc0a3290 0x7f99dc0a5750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99f0109ab0 0x7f99f019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0104460 0x7f99f019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 --2- 172.21.15.49:0/785413593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99f0103aa0 0x7f99f019a3a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:40.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 -- 172.21.15.49:0/785413593 >> 172.21.15.49:0/785413593 conn(0x7f99f00ff570 msgr2=0x7f99f00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:40.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 -- 172.21.15.49:0/785413593 shutdown_connections 2024-03-26T18:17:40.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:40.958+0000 7f99f587e640 1 -- 172.21.15.49:0/785413593 wait complete. 2024-03-26T18:17:40.944 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-26T18:17:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 -- 172.21.15.49:0/4082150145 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1c8101870 msgr2=0x7fd1c8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 --2- 172.21.15.49:0/4082150145 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1c8101870 0x7fd1c8101cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009890 tx=0x7fd1b002bb70 comp rx=0 tx=0).stop 2024-03-26T18:17:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 -- 172.21.15.49:0/4082150145 shutdown_connections 2024-03-26T18:17:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 --2- 172.21.15.49:0/4082150145 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1c8101870 0x7fd1c8101cf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 --2- 172.21.15.49:0/4082150145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c8102230 0x7fd1c810e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 --2- 172.21.15.49:0/4082150145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1c8107880 0x7fd1c8107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 -- 172.21.15.49:0/4082150145 >> 172.21.15.49:0/4082150145 conn(0x7fd1c80fd380 msgr2=0x7fd1c80ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:41.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 -- 172.21.15.49:0/4082150145 shutdown_connections 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 -- 172.21.15.49:0/4082150145 wait complete. 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 Processor -- start 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.081+0000 7fd1cda19640 1 -- start start 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cda19640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1c8101870 0x7fd1c8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cda19640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1c8102230 0x7fd1c8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cda19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c8107880 0x7fd1c819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cda19640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c8111c90 con 0x7fd1c8107880 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cda19640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c8111b10 con 0x7fd1c8101870 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cda19640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c8111e10 con 0x7fd1c8102230 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1c77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c8107880 0x7fd1c819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1c77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c8107880 0x7fd1c819c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35142/0 (socket says 172.21.15.49:35142) 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1c77fe640 1 -- 172.21.15.49:0/1554703757 learned_addr learned my addr 172.21.15.49:0/1554703757 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1c6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1c8101870 0x7fd1c8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1bffff640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1c8102230 0x7fd1c8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1c77fe640 1 -- 172.21.15.49:0/1554703757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1c8102230 msgr2=0x7fd1c8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1c77fe640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1c8102230 0x7fd1c8198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1c77fe640 1 -- 172.21.15.49:0/1554703757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1c8101870 msgr2=0x7fd1c8198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1c77fe640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1c8101870 0x7fd1c8198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1c77fe640 1 -- 172.21.15.49:0/1554703757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c819d120 con 0x7fd1c8107880 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1c77fe640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c8107880 0x7fd1c819c9f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000efc0 tx=0x7fd1c000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cca17640 1 -- 172.21.15.49:0/1554703757 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0003970 con 0x7fd1c8107880 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cca17640 1 -- 172.21.15.49:0/1554703757 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c0004020 con 0x7fd1c8107880 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cda19640 1 -- 172.21.15.49:0/1554703757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c819d410 con 0x7fd1c8107880 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cca17640 1 -- 172.21.15.49:0/1554703757 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000b9f0 con 0x7fd1c8107880 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.082+0000 7fd1cda19640 1 -- 172.21.15.49:0/1554703757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c81a4ca0 con 0x7fd1c8107880 2024-03-26T18:17:41.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.083+0000 7fd1cda19640 1 -- 172.21.15.49:0/1554703757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c804f620 con 0x7fd1c8107880 2024-03-26T18:17:41.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.085+0000 7fd1cca17640 1 -- 172.21.15.49:0/1554703757 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7fd1c001d020 con 0x7fd1c8107880 2024-03-26T18:17:41.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.085+0000 7fd1cca17640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd1b40a3130 0x7fd1b40a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.085+0000 7fd1c6ffd640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd1b40a3130 0x7fd1b40a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.085+0000 7fd1cca17640 1 -- 172.21.15.49:0/1554703757 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7fd1c00cfa60 con 0x7fd1c8107880 2024-03-26T18:17:41.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.086+0000 7fd1c6ffd640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd1b40a3130 0x7fd1b40a55f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009b50 tx=0x7fd1b8005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:41.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.090+0000 7fd1cca17640 1 -- 172.21.15.49:0/1554703757 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd1c009c190 con 0x7fd1c8107880 2024-03-26T18:17:41.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.313+0000 7fd1cda19640 1 -- 172.21.15.49:0/1554703757 --> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd1c8061d00 con 0x7fd1b40a3130 2024-03-26T18:17:41.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.319+0000 7fd1cca17640 1 -- 172.21.15.49:0/1554703757 <== mgr.15516 v2:172.21.15.49:6800/2879593470 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd1c8061d00 con 0x7fd1b40a3130 2024-03-26T18:17:41.294 INFO:teuthology.orchestra.run.smithi049.stdout:/ 1m 6m3h 2024-03-26T18:17:41.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 -- 172.21.15.49:0/1554703757 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd1b40a3130 msgr2=0x7fd1b40a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd1b40a3130 0x7fd1b40a55f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009b50 tx=0x7fd1b8005f90 comp rx=0 tx=0).stop 2024-03-26T18:17:41.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 -- 172.21.15.49:0/1554703757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c8107880 msgr2=0x7fd1c819c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c8107880 0x7fd1c819c9f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000efc0 tx=0x7fd1c000c800 comp rx=0 tx=0).stop 2024-03-26T18:17:41.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 -- 172.21.15.49:0/1554703757 shutdown_connections 2024-03-26T18:17:41.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd1b40a3130 0x7fd1b40a55f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1c8102230 0x7fd1c8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1c8107880 0x7fd1c819c9f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 --2- 172.21.15.49:0/1554703757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1c8101870 0x7fd1c8198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 -- 172.21.15.49:0/1554703757 >> 172.21.15.49:0/1554703757 conn(0x7fd1c80fd380 msgr2=0x7fd1c80fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:41.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 -- 172.21.15.49:0/1554703757 shutdown_connections 2024-03-26T18:17:41.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.321+0000 7fd1cda19640 1 -- 172.21.15.49:0/1554703757 wait complete. 2024-03-26T18:17:41.308 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-26T18:17:41.320 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:17:41.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.646+0000 7f15f0ac6640 1 -- 172.21.15.49:0/4241894226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec100760 msgr2=0x7f15ec100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.646+0000 7f15f0ac6640 1 --2- 172.21.15.49:0/4241894226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec100760 0x7f15ec100be0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009930 tx=0x7f15d402bc30 comp rx=0 tx=0).stop 2024-03-26T18:17:41.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.646+0000 7f15f0ac6640 1 -- 172.21.15.49:0/4241894226 shutdown_connections 2024-03-26T18:17:41.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.646+0000 7f15f0ac6640 1 --2- 172.21.15.49:0/4241894226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec101120 0x7f15ec10d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.646+0000 7f15f0ac6640 1 --2- 172.21.15.49:0/4241894226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec100760 0x7f15ec100be0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.646+0000 7f15f0ac6640 1 --2- 172.21.15.49:0/4241894226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15ec106770 0x7f15ec106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.646+0000 7f15f0ac6640 1 -- 172.21.15.49:0/4241894226 >> 172.21.15.49:0/4241894226 conn(0x7f15ec076b40 msgr2=0x7f15ec076f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:41.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.646+0000 7f15f0ac6640 1 -- 172.21.15.49:0/4241894226 shutdown_connections 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.646+0000 7f15f0ac6640 1 -- 172.21.15.49:0/4241894226 wait complete. 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15f0ac6640 1 Processor -- start 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15f0ac6640 1 -- start start 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15f0ac6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15ec100760 0x7f15ec0fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15f0ac6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec101120 0x7f15ec0fe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15f0ac6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec106770 0x7f15ec0fea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15f0ac6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec110b80 con 0x7f15ec106770 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15f0ac6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec110a00 con 0x7f15ec101120 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15f0ac6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec110d00 con 0x7f15ec100760 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec101120 0x7f15ec0fe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15ead9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec106770 0x7f15ec0fea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.647+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec101120 0x7f15ec0fe4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.49:52024/0 (socket says 172.21.15.49:52024) 2024-03-26T18:17:41.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15e3fff640 1 -- 172.21.15.49:0/1691025164 learned_addr learned my addr 172.21.15.49:0/1691025164 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15ea59c640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15ec100760 0x7f15ec0fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15e3fff640 1 -- 172.21.15.49:0/1691025164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15ec100760 msgr2=0x7f15ec0fdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15e3fff640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15ec100760 0x7f15ec0fdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15e3fff640 1 -- 172.21.15.49:0/1691025164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec106770 msgr2=0x7f15ec0fea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15e3fff640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec106770 0x7f15ec0fea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15e3fff640 1 -- 172.21.15.49:0/1691025164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec1a6690 con 0x7f15ec101120 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15ea59c640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15ec100760 0x7f15ec0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15ead9d640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec106770 0x7f15ec0fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15e3fff640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec101120 0x7f15ec0fe4d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009230 tx=0x7f15d402cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15e37fe640 1 -- 172.21.15.49:0/1691025164 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4041070 con 0x7f15ec101120 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15f0ac6640 1 -- 172.21.15.49:0/1691025164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ec1a6920 con 0x7f15ec101120 2024-03-26T18:17:41.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15e37fe640 1 -- 172.21.15.49:0/1691025164 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d4004450 con 0x7f15ec101120 2024-03-26T18:17:41.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15e37fe640 1 -- 172.21.15.49:0/1691025164 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d404b060 con 0x7f15ec101120 2024-03-26T18:17:41.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.648+0000 7f15f0ac6640 1 -- 172.21.15.49:0/1691025164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ec1a6e60 con 0x7f15ec101120 2024-03-26T18:17:41.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.649+0000 7f15f0ac6640 1 -- 172.21.15.49:0/1691025164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ec101ea0 con 0x7f15ec101120 2024-03-26T18:17:41.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.651+0000 7f15e37fe640 1 -- 172.21.15.49:0/1691025164 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f15d4044070 con 0x7f15ec101120 2024-03-26T18:17:41.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.651+0000 7f15e37fe640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f15d80a3090 0x7f15d80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.652+0000 7f15ea59c640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f15d80a3090 0x7f15d80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.652+0000 7f15e37fe640 1 -- 172.21.15.49:0/1691025164 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f15d40f16f0 con 0x7f15ec101120 2024-03-26T18:17:41.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.652+0000 7f15ea59c640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f15d80a3090 0x7f15d80a5550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009f40 tx=0x7f15dc005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:41.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.655+0000 7f15e37fe640 1 -- 172.21.15.49:0/1691025164 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f15d40bd8e0 con 0x7f15ec101120 2024-03-26T18:17:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.842+0000 7f15f0ac6640 1 -- 172.21.15.49:0/1691025164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15ec109d00 con 0x7f15ec101120 2024-03-26T18:17:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.844+0000 7f15e37fe640 1 -- 172.21.15.49:0/1691025164 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+25237 (secure 0 0 0) 0x7f15d40a6e20 con 0x7f15ec101120 2024-03-26T18:17:41.819 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:41.819 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":113,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T18:17:17.708542+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":112,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":112,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6801/3127469495":"2024-03-27T18:02:05.919612+0000","172.21.15.49:6800/3127469495":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/1873884295":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/1574907427":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1579518314":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/625009684":"2024-03-27T18:02:05.919612+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]},{"pool":4,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:17:41.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 -- 172.21.15.49:0/1691025164 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f15d80a3090 msgr2=0x7f15d80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f15d80a3090 0x7f15d80a5550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009f40 tx=0x7f15dc005f90 comp rx=0 tx=0).stop 2024-03-26T18:17:41.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 -- 172.21.15.49:0/1691025164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec101120 msgr2=0x7f15ec0fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec101120 0x7f15ec0fe4d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009230 tx=0x7f15d402cb30 comp rx=0 tx=0).stop 2024-03-26T18:17:41.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 -- 172.21.15.49:0/1691025164 shutdown_connections 2024-03-26T18:17:41.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f15d80a3090 0x7f15d80a5550 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15ec106770 0x7f15ec0fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ec101120 0x7f15ec0fe4d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 --2- 172.21.15.49:0/1691025164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15ec100760 0x7f15ec0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 -- 172.21.15.49:0/1691025164 >> 172.21.15.49:0/1691025164 conn(0x7f15ec076b40 msgr2=0x7f15ec06a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 -- 172.21.15.49:0/1691025164 shutdown_connections 2024-03-26T18:17:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.846+0000 7f15f0ac6640 1 -- 172.21.15.49:0/1691025164 wait complete. 2024-03-26T18:17:41.833 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:59:45.382542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:59:46.221139+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 112, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T17:59:49.890383+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 112, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T18:17:41.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:17:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.970+0000 7f76c92c0640 1 -- 172.21.15.49:0/3306587202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76c4105660 msgr2=0x7f76c4105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.970+0000 7f76c92c0640 1 --2- 172.21.15.49:0/3306587202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76c4105660 0x7f76c4105a40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009900 tx=0x7f76b802bcd0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.970+0000 7f76c92c0640 1 -- 172.21.15.49:0/3306587202 shutdown_connections 2024-03-26T18:17:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.970+0000 7f76c92c0640 1 --2- 172.21.15.49:0/3306587202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c4101120 0x7f76c410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.970+0000 7f76c92c0640 1 --2- 172.21.15.49:0/3306587202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76c4100780 0x7f76c4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.970+0000 7f76c92c0640 1 --2- 172.21.15.49:0/3306587202 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76c4105660 0x7f76c4105a40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.970+0000 7f76c92c0640 1 -- 172.21.15.49:0/3306587202 >> 172.21.15.49:0/3306587202 conn(0x7f76c4077850 msgr2=0x7f76c4077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.970+0000 7f76c92c0640 1 -- 172.21.15.49:0/3306587202 shutdown_connections 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.970+0000 7f76c92c0640 1 -- 172.21.15.49:0/3306587202 wait complete. 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.971+0000 7f76c92c0640 1 Processor -- start 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.971+0000 7f76c92c0640 1 -- start start 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.971+0000 7f76c92c0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76c4100780 0x7f76c4102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.971+0000 7f76c92c0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c4101120 0x7f76c4102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.971+0000 7f76c92c0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76c4105660 0x7f76c4102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.971+0000 7f76c92c0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4110ba0 con 0x7f76c4100780 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.971+0000 7f76c92c0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4110a20 con 0x7f76c4105660 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.971+0000 7f76c92c0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4110d20 con 0x7f76c4101120 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76c4100780 0x7f76c4102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76c4100780 0x7f76c4102300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35176/0 (socket says 172.21.15.49:35176) 2024-03-26T18:17:41.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c2ffd640 1 -- 172.21.15.49:0/189710521 learned_addr learned my addr 172.21.15.49:0/189710521 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c27fc640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c4101120 0x7f76c4102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c37fe640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76c4105660 0x7f76c4102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c2ffd640 1 -- 172.21.15.49:0/189710521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c4101120 msgr2=0x7f76c4102840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c2ffd640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c4101120 0x7f76c4102840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c2ffd640 1 -- 172.21.15.49:0/189710521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76c4105660 msgr2=0x7f76c4102d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c2ffd640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76c4105660 0x7f76c4102d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c2ffd640 1 -- 172.21.15.49:0/189710521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c41a66a0 con 0x7f76c4100780 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c27fc640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c4101120 0x7f76c4102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c37fe640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76c4105660 0x7f76c4102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c2ffd640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76c4100780 0x7f76c4102300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f76b8000c00 tx=0x7f76b8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76abfff640 1 -- 172.21.15.49:0/189710521 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8042020 con 0x7f76c4100780 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c92c0640 1 -- 172.21.15.49:0/189710521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c41a6930 con 0x7f76c4100780 2024-03-26T18:17:41.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76abfff640 1 -- 172.21.15.49:0/189710521 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b80054e0 con 0x7f76c4100780 2024-03-26T18:17:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76abfff640 1 -- 172.21.15.49:0/189710521 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8003af0 con 0x7f76c4100780 2024-03-26T18:17:41.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.972+0000 7f76c92c0640 1 -- 172.21.15.49:0/189710521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c41a6ed0 con 0x7f76c4100780 2024-03-26T18:17:41.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.974+0000 7f76c92c0640 1 -- 172.21.15.49:0/189710521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c4067e20 con 0x7f76c4100780 2024-03-26T18:17:41.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.974+0000 7f76abfff640 1 -- 172.21.15.49:0/189710521 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f76b8045020 con 0x7f76c4100780 2024-03-26T18:17:41.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.975+0000 7f76abfff640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f76ac0a3090 0x7f76ac0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:41.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.975+0000 7f76c27fc640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f76ac0a3090 0x7f76ac0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:41.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.975+0000 7f76abfff640 1 -- 172.21.15.49:0/189710521 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f76b80f1670 con 0x7f76c4100780 2024-03-26T18:17:41.952 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.976+0000 7f76c27fc640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f76ac0a3090 0x7f76ac0a5550 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76c4103d90 tx=0x7f76b4005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:41.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:41.980+0000 7f76abfff640 1 -- 172.21.15.49:0/189710521 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f76b804c020 con 0x7f76c4100780 2024-03-26T18:17:42.065 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:41 smithi049 ceph-mon[32331]: pgmap v488: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 6.3 MiB/s wr, 669 op/s 2024-03-26T18:17:42.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:41 smithi171 ceph-mon[36033]: pgmap v488: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 6.3 MiB/s wr, 669 op/s 2024-03-26T18:17:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.194+0000 7f76c92c0640 1 -- 172.21.15.49:0/189710521 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76c4109c40 con 0x7f76c4100780 2024-03-26T18:17:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.194+0000 7f76abfff640 1 -- 172.21.15.49:0/189710521 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f76b80bdeb0 con 0x7f76c4100780 2024-03-26T18:17:42.170 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T18:17:42.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.196+0000 7f76c92c0640 1 -- 172.21.15.49:0/189710521 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f76ac0a3090 msgr2=0x7f76ac0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.196+0000 7f76c92c0640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f76ac0a3090 0x7f76ac0a5550 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76c4103d90 tx=0x7f76b4005f90 comp rx=0 tx=0).stop 2024-03-26T18:17:42.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.196+0000 7f76c92c0640 1 -- 172.21.15.49:0/189710521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76c4100780 msgr2=0x7f76c4102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.196+0000 7f76c92c0640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76c4100780 0x7f76c4102300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f76b8000c00 tx=0x7f76b8004360 comp rx=0 tx=0).stop 2024-03-26T18:17:42.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.196+0000 7f76c92c0640 1 -- 172.21.15.49:0/189710521 shutdown_connections 2024-03-26T18:17:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.197+0000 7f76c92c0640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f76ac0a3090 0x7f76ac0a5550 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.197+0000 7f76c92c0640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c4101120 0x7f76c4102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.197+0000 7f76c92c0640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76c4105660 0x7f76c4102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.197+0000 7f76c92c0640 1 --2- 172.21.15.49:0/189710521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f76c4100780 0x7f76c4102300 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.197+0000 7f76c92c0640 1 -- 172.21.15.49:0/189710521 >> 172.21.15.49:0/189710521 conn(0x7f76c4077850 msgr2=0x7f76c4069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.197+0000 7f76c92c0640 1 -- 172.21.15.49:0/189710521 shutdown_connections 2024-03-26T18:17:42.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.197+0000 7f76c92c0640 1 -- 172.21.15.49:0/189710521 wait complete. 2024-03-26T18:17:42.183 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:17:42.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:41 smithi115 ceph-mon[36174]: pgmap v488: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 6.3 MiB/s wr, 669 op/s 2024-03-26T18:17:42.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.320+0000 7faf872f1640 1 -- 172.21.15.49:0/3463676018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf80106740 msgr2=0x7faf80106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.320+0000 7faf872f1640 1 --2- 172.21.15.49:0/3463676018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf80106740 0x7faf80106b20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009930 tx=0x7faf7c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:17:42.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.320+0000 7faf872f1640 1 -- 172.21.15.49:0/3463676018 shutdown_connections 2024-03-26T18:17:42.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.320+0000 7faf872f1640 1 --2- 172.21.15.49:0/3463676018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf801010f0 0x7faf8010d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.320+0000 7faf872f1640 1 --2- 172.21.15.49:0/3463676018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf80100730 0x7faf80100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.320+0000 7faf872f1640 1 --2- 172.21.15.49:0/3463676018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf80106740 0x7faf80106b20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.320+0000 7faf872f1640 1 -- 172.21.15.49:0/3463676018 >> 172.21.15.49:0/3463676018 conn(0x7faf800778c0 msgr2=0x7faf80077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:42.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.320+0000 7faf872f1640 1 -- 172.21.15.49:0/3463676018 shutdown_connections 2024-03-26T18:17:42.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.320+0000 7faf872f1640 1 -- 172.21.15.49:0/3463676018 wait complete. 2024-03-26T18:17:42.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf872f1640 1 Processor -- start 2024-03-26T18:17:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf872f1640 1 -- start start 2024-03-26T18:17:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf872f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf80100730 0x7faf80198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf872f1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf801010f0 0x7faf80198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf872f1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf80106740 0x7faf8019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf872f1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7faf80110ba0 con 0x7faf801010f0 2024-03-26T18:17:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf872f1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7faf80110a20 con 0x7faf80106740 2024-03-26T18:17:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf872f1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faf80110d20 con 0x7faf80100730 2024-03-26T18:17:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8488c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf801010f0 0x7faf80198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8488c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf801010f0 0x7faf80198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35196/0 (socket says 172.21.15.49:35196) 2024-03-26T18:17:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8488c640 1 -- 172.21.15.49:0/2163452475 learned_addr learned my addr 172.21.15.49:0/2163452475 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8508d640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf80100730 0x7faf80198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8588e640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf80106740 0x7faf8019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8488c640 1 -- 172.21.15.49:0/2163452475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf80100730 msgr2=0x7faf80198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8488c640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf80100730 0x7faf80198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8488c640 1 -- 172.21.15.49:0/2163452475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf80106740 msgr2=0x7faf8019c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8488c640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf80106740 0x7faf8019c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8488c640 1 -- 172.21.15.49:0/2163452475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8019d120 con 0x7faf801010f0 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.321+0000 7faf8488c640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf801010f0 0x7faf80198670 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faf7000e9f0 tx=0x7faf7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.322+0000 7faf767fc640 1 -- 172.21.15.49:0/2163452475 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7000cd80 con 0x7faf801010f0 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.322+0000 7faf872f1640 1 -- 172.21.15.49:0/2163452475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf8019d410 con 0x7faf801010f0 2024-03-26T18:17:42.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.322+0000 7faf767fc640 1 -- 172.21.15.49:0/2163452475 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf70010910 con 0x7faf801010f0 2024-03-26T18:17:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.322+0000 7faf767fc640 1 -- 172.21.15.49:0/2163452475 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700188f0 con 0x7faf801010f0 2024-03-26T18:17:42.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.322+0000 7faf872f1640 1 -- 172.21.15.49:0/2163452475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf801a4ca0 con 0x7faf801010f0 2024-03-26T18:17:42.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.323+0000 7faf872f1640 1 -- 172.21.15.49:0/2163452475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf800686b0 con 0x7faf801010f0 2024-03-26T18:17:42.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.324+0000 7faf767fc640 1 -- 172.21.15.49:0/2163452475 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7faf70010ac0 con 0x7faf801010f0 2024-03-26T18:17:42.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.325+0000 7faf767fc640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7faf6c0a3090 0x7faf6c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.325+0000 7faf8508d640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7faf6c0a3090 0x7faf6c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.325+0000 7faf767fc640 1 -- 172.21.15.49:0/2163452475 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7faf700d02e0 con 0x7faf801010f0 2024-03-26T18:17:42.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.325+0000 7faf8508d640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7faf6c0a3090 0x7faf6c0a5550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009900 tx=0x7faf7c02df90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:42.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.328+0000 7faf767fc640 1 -- 172.21.15.49:0/2163452475 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faf7009c3c0 con 0x7faf801010f0 2024-03-26T18:17:42.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.512+0000 7faf872f1640 1 -- 172.21.15.49:0/2163452475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faf801a4f90 con 0x7faf801010f0 2024-03-26T18:17:42.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.513+0000 7faf767fc640 1 -- 172.21.15.49:0/2163452475 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7faf700a0460 con 0x7faf801010f0 2024-03-26T18:17:42.488 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:17:42.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 -- 172.21.15.49:0/2163452475 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7faf6c0a3090 msgr2=0x7faf6c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7faf6c0a3090 0x7faf6c0a5550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009900 tx=0x7faf7c02df90 comp rx=0 tx=0).stop 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 -- 172.21.15.49:0/2163452475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf801010f0 msgr2=0x7faf80198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf801010f0 0x7faf80198670 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faf7000e9f0 tx=0x7faf7000eeb0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 -- 172.21.15.49:0/2163452475 shutdown_connections 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7faf6c0a3090 0x7faf6c0a5550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf80106740 0x7faf8019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7faf801010f0 0x7faf80198670 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 --2- 172.21.15.49:0/2163452475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf80100730 0x7faf80198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 -- 172.21.15.49:0/2163452475 >> 172.21.15.49:0/2163452475 conn(0x7faf800778c0 msgr2=0x7faf800ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 -- 172.21.15.49:0/2163452475 shutdown_connections 2024-03-26T18:17:42.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.515+0000 7faf872f1640 1 -- 172.21.15.49:0/2163452475 wait complete. 2024-03-26T18:17:42.502 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:17:42.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.641+0000 7f46fae9a640 1 -- 172.21.15.49:0/3028540854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f4100730 msgr2=0x7f46f4100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.641+0000 7f46fae9a640 1 --2- 172.21.15.49:0/3028540854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f4100730 0x7f46f4100bb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009930 tx=0x7f46e802bc20 comp rx=0 tx=0).stop 2024-03-26T18:17:42.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.641+0000 7f46fae9a640 1 -- 172.21.15.49:0/3028540854 shutdown_connections 2024-03-26T18:17:42.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.641+0000 7f46fae9a640 1 --2- 172.21.15.49:0/3028540854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46f41010f0 0x7f46f410d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.641+0000 7f46fae9a640 1 --2- 172.21.15.49:0/3028540854 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f4100730 0x7f46f4100bb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.641+0000 7f46fae9a640 1 --2- 172.21.15.49:0/3028540854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f4106740 0x7f46f4106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.641+0000 7f46fae9a640 1 -- 172.21.15.49:0/3028540854 >> 172.21.15.49:0/3028540854 conn(0x7f46f40778c0 msgr2=0x7f46f4077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.642+0000 7f46fae9a640 1 -- 172.21.15.49:0/3028540854 shutdown_connections 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.642+0000 7f46fae9a640 1 -- 172.21.15.49:0/3028540854 wait complete. 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.642+0000 7f46fae9a640 1 Processor -- start 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.642+0000 7f46fae9a640 1 -- start start 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.642+0000 7f46fae9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f4100730 0x7f46f41970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.642+0000 7f46fae9a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f41010f0 0x7f46f4197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.642+0000 7f46fae9a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46f4106740 0x7f46f419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.642+0000 7f46fae9a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4110ba0 con 0x7f46f4100730 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.642+0000 7f46fae9a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4110a20 con 0x7f46f4106740 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.642+0000 7f46fae9a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4110d20 con 0x7f46f41010f0 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f8c36640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f4100730 0x7f46f41970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f9437640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46f4106740 0x7f46f419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f8c36640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f4100730 0x7f46f41970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35210/0 (socket says 172.21.15.49:35210) 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f8c36640 1 -- 172.21.15.49:0/3237755595 learned_addr learned my addr 172.21.15.49:0/3237755595 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:17:42.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f3fff640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f41010f0 0x7f46f4197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f8c36640 1 -- 172.21.15.49:0/3237755595 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f41010f0 msgr2=0x7f46f4197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f8c36640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f41010f0 0x7f46f4197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f8c36640 1 -- 172.21.15.49:0/3237755595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46f4106740 msgr2=0x7f46f419b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f8c36640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46f4106740 0x7f46f419b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f8c36640 1 -- 172.21.15.49:0/3237755595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f419c0b0 con 0x7f46f4100730 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f9437640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46f4106740 0x7f46f419b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f8c36640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f4100730 0x7f46f41970c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f46e000eb00 tx=0x7f46e000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f1ffb640 1 -- 172.21.15.49:0/3237755595 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e000bc30 con 0x7f46f4100730 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f1ffb640 1 -- 172.21.15.49:0/3237755595 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46e000bdd0 con 0x7f46f4100730 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46fae9a640 1 -- 172.21.15.49:0/3237755595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f419c3a0 con 0x7f46f4100730 2024-03-26T18:17:42.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46f1ffb640 1 -- 172.21.15.49:0/3237755595 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0017410 con 0x7f46f4100730 2024-03-26T18:17:42.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.643+0000 7f46fae9a640 1 -- 172.21.15.49:0/3237755595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f4068a20 con 0x7f46f4100730 2024-03-26T18:17:42.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.644+0000 7f46fae9a640 1 -- 172.21.15.49:0/3237755595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f40686b0 con 0x7f46f4100730 2024-03-26T18:17:42.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.646+0000 7f46f1ffb640 1 -- 172.21.15.49:0/3237755595 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f46e0005000 con 0x7f46f4100730 2024-03-26T18:17:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.646+0000 7f46f1ffb640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f46e40a3110 0x7f46e40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.646+0000 7f46f1ffb640 1 -- 172.21.15.49:0/3237755595 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f46e00d1850 con 0x7f46f4100730 2024-03-26T18:17:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.646+0000 7f46f3fff640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f46e40a3110 0x7f46e40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.647+0000 7f46f3fff640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f46e40a3110 0x7f46e40a55d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009900 tx=0x7f46e8005a20 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:42.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.650+0000 7f46f1ffb640 1 -- 172.21.15.49:0/3237755595 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f46e009d9b0 con 0x7f46f4100730 2024-03-26T18:17:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.835+0000 7f46fae9a640 1 -- 172.21.15.49:0/3237755595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f46f4069440 con 0x7f46f4100730 2024-03-26T18:17:42.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.835+0000 7f46f1ffb640 1 -- 172.21.15.49:0/3237755595 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7f46e00a1a50 con 0x7f46f4100730 2024-03-26T18:17:42.811 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-26T18:17:42.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 -- 172.21.15.49:0/3237755595 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f46e40a3110 msgr2=0x7f46e40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f46e40a3110 0x7f46e40a55d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009900 tx=0x7f46e8005a20 comp rx=0 tx=0).stop 2024-03-26T18:17:42.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 -- 172.21.15.49:0/3237755595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f4100730 msgr2=0x7f46f41970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f4100730 0x7f46f41970c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f46e000eb00 tx=0x7f46e000ef00 comp rx=0 tx=0).stop 2024-03-26T18:17:42.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 -- 172.21.15.49:0/3237755595 shutdown_connections 2024-03-26T18:17:42.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f46e40a3110 0x7f46e40a55d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46f41010f0 0x7f46f4197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46f4106740 0x7f46f419b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 --2- 172.21.15.49:0/3237755595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46f4100730 0x7f46f41970c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 -- 172.21.15.49:0/3237755595 >> 172.21.15.49:0/3237755595 conn(0x7f46f40778c0 msgr2=0x7f46f4102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:42.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 -- 172.21.15.49:0/3237755595 shutdown_connections 2024-03-26T18:17:42.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.837+0000 7f46fae9a640 1 -- 172.21.15.49:0/3237755595 wait complete. 2024-03-26T18:17:42.826 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T18:17:42.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.962+0000 7f003d0ea640 1 -- 172.21.15.49:0/3224290174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00381010f0 msgr2=0x7f003810d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.962+0000 7f003d0ea640 1 --2- 172.21.15.49:0/3224290174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00381010f0 0x7f003810d600 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0030009cd0 tx=0x7f003002c0f0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 -- 172.21.15.49:0/3224290174 shutdown_connections 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 --2- 172.21.15.49:0/3224290174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00381010f0 0x7f003810d600 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 --2- 172.21.15.49:0/3224290174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0038100730 0x7f0038100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 --2- 172.21.15.49:0/3224290174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0038106740 0x7f0038106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 -- 172.21.15.49:0/3224290174 >> 172.21.15.49:0/3224290174 conn(0x7f00380778c0 msgr2=0x7f0038077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 -- 172.21.15.49:0/3224290174 shutdown_connections 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 -- 172.21.15.49:0/3224290174 wait complete. 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 Processor -- start 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 -- start start 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038100730 0x7f0038102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00381010f0 0x7f0038103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0038106740 0x7f0038103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0038110b50 con 0x7f0038100730 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f00381109d0 con 0x7f0038106740 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.963+0000 7f003d0ea640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0038110cd0 con 0x7f00381010f0 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f0036d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038100730 0x7f0038102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f0036d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038100730 0x7f0038102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35236/0 (socket says 172.21.15.49:35236) 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f0036d9d640 1 -- 172.21.15.49:0/2715147551 learned_addr learned my addr 172.21.15.49:0/2715147551 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f003759e640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0038106740 0x7f0038103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f003659c640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00381010f0 0x7f0038103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f0036d9d640 1 -- 172.21.15.49:0/2715147551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00381010f0 msgr2=0x7f0038103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f0036d9d640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00381010f0 0x7f0038103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f0036d9d640 1 -- 172.21.15.49:0/2715147551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0038106740 msgr2=0x7f0038103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:42.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f0036d9d640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0038106740 0x7f0038103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:42.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f0036d9d640 1 -- 172.21.15.49:0/2715147551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00381a6610 con 0x7f0038100730 2024-03-26T18:17:42.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f0036d9d640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038100730 0x7f0038102cd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f002c00e9f0 tx=0x7f002c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:42.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f001ffff640 1 -- 172.21.15.49:0/2715147551 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c00cd80 con 0x7f0038100730 2024-03-26T18:17:42.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f003d0ea640 1 -- 172.21.15.49:0/2715147551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00381a6900 con 0x7f0038100730 2024-03-26T18:17:42.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f001ffff640 1 -- 172.21.15.49:0/2715147551 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f002c005500 con 0x7f0038100730 2024-03-26T18:17:42.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f001ffff640 1 -- 172.21.15.49:0/2715147551 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c010690 con 0x7f0038100730 2024-03-26T18:17:42.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.964+0000 7f003d0ea640 1 -- 172.21.15.49:0/2715147551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00381a6e00 con 0x7f0038100730 2024-03-26T18:17:42.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.965+0000 7f003d0ea640 1 -- 172.21.15.49:0/2715147551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00380686b0 con 0x7f0038100730 2024-03-26T18:17:42.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.966+0000 7f001ffff640 1 -- 172.21.15.49:0/2715147551 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f002c0046c0 con 0x7f0038100730 2024-03-26T18:17:42.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.967+0000 7f001ffff640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f00200a3090 0x7f00200a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:42.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.967+0000 7f001ffff640 1 -- 172.21.15.49:0/2715147551 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f002c0cff20 con 0x7f0038100730 2024-03-26T18:17:42.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.967+0000 7f003659c640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f00200a3090 0x7f00200a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:42.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.967+0000 7f003659c640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f00200a3090 0x7f00200a5550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f00280095f0 tx=0x7f0028005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:42.947 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:42.972+0000 7f001ffff640 1 -- 172.21.15.49:0/2715147551 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f002c09c650 con 0x7f0038100730 2024-03-26T18:17:43.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:42 smithi171 ceph-mon[36033]: from='client.25529 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:17:43.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:42 smithi171 ceph-mon[36033]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:17:43.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:42 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1691025164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:17:43.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:42 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/189710521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:17:43.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:42 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2163452475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:17:43.130 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:42 smithi115 ceph-mon[36174]: from='client.25529 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:17:43.130 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:42 smithi115 ceph-mon[36174]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:17:43.130 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:42 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1691025164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:17:43.130 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:42 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/189710521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:17:43.130 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:42 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2163452475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:17:43.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.155+0000 7f003d0ea640 1 -- 172.21.15.49:0/2715147551 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0038109cb0 con 0x7f0038100730 2024-03-26T18:17:43.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.155+0000 7f001ffff640 1 -- 172.21.15.49:0/2715147551 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v113) ==== 103+0+11 (secure 0 0 0) 0x7f002c09bf50 con 0x7f0038100730 2024-03-26T18:17:43.131 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 -- 172.21.15.49:0/2715147551 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f00200a3090 msgr2=0x7f00200a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f00200a3090 0x7f00200a5550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f00280095f0 tx=0x7f0028005f90 comp rx=0 tx=0).stop 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 -- 172.21.15.49:0/2715147551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038100730 msgr2=0x7f0038102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038100730 0x7f0038102cd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f002c00e9f0 tx=0x7f002c00edf0 comp rx=0 tx=0).stop 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 -- 172.21.15.49:0/2715147551 shutdown_connections 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f00200a3090 0x7f00200a5550 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f00381010f0 0x7f0038103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0038106740 0x7f0038103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 --2- 172.21.15.49:0/2715147551 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0038100730 0x7f0038102cd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 -- 172.21.15.49:0/2715147551 >> 172.21.15.49:0/2715147551 conn(0x7f00380778c0 msgr2=0x7f0038068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 -- 172.21.15.49:0/2715147551 shutdown_connections 2024-03-26T18:17:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.157+0000 7f003d0ea640 1 -- 172.21.15.49:0/2715147551 wait complete. 2024-03-26T18:17:43.146 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:42 smithi049 ceph-mon[32331]: from='client.25529 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:17:43.146 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:42 smithi049 ceph-mon[32331]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T18:17:43.146 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:42 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1691025164' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:17:43.146 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:42 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/189710521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:17:43.146 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:42 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2163452475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:17:43.147 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-26T18:17:43.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.280+0000 7f93729b3640 1 -- 172.21.15.49:0/3904644936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f936c1067d0 msgr2=0x7f936c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:43.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.280+0000 7f93729b3640 1 --2- 172.21.15.49:0/3904644936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f936c1067d0 0x7f936c106bb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f935c009900 tx=0x7f935c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:17:43.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.280+0000 7f93729b3640 1 -- 172.21.15.49:0/3904644936 shutdown_connections 2024-03-26T18:17:43.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.280+0000 7f93729b3640 1 --2- 172.21.15.49:0/3904644936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f936c1007c0 0x7f936c100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:43.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.280+0000 7f93729b3640 1 --2- 172.21.15.49:0/3904644936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c101180 0x7f936c10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:43.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.280+0000 7f93729b3640 1 --2- 172.21.15.49:0/3904644936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f936c1067d0 0x7f936c106bb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:43.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.280+0000 7f93729b3640 1 -- 172.21.15.49:0/3904644936 >> 172.21.15.49:0/3904644936 conn(0x7f936c0778c0 msgr2=0x7f936c077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:43.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.280+0000 7f93729b3640 1 -- 172.21.15.49:0/3904644936 shutdown_connections 2024-03-26T18:17:43.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f93729b3640 1 -- 172.21.15.49:0/3904644936 wait complete. 2024-03-26T18:17:43.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f93729b3640 1 Processor -- start 2024-03-26T18:17:43.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f93729b3640 1 -- start start 2024-03-26T18:17:43.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f93729b3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f936c1007c0 0x7f936c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:43.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f93729b3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f936c101180 0x7f936c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:43.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f93729b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c1067d0 0x7f936c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:43.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f93729b3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f936c110be0 con 0x7f936c1067d0 2024-03-26T18:17:43.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f93729b3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f936c110a60 con 0x7f936c1007c0 2024-03-26T18:17:43.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f93729b3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f936c110d60 con 0x7f936c101180 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f9370f50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c1067d0 0x7f936c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f9370f50640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c1067d0 0x7f936c197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35262/0 (socket says 172.21.15.49:35262) 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f9370f50640 1 -- 172.21.15.49:0/4082503021 learned_addr learned my addr 172.21.15.49:0/4082503021 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f936b7fe640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f936c101180 0x7f936c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.281+0000 7f936bfff640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f936c1007c0 0x7f936c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f9370f50640 1 -- 172.21.15.49:0/4082503021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f936c101180 msgr2=0x7f936c1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f9370f50640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f936c101180 0x7f936c1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f9370f50640 1 -- 172.21.15.49:0/4082503021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f936c1007c0 msgr2=0x7f936c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f9370f50640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f936c1007c0 0x7f936c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f9370f50640 1 -- 172.21.15.49:0/4082503021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c1a6620 con 0x7f936c1067d0 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f9370f50640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c1067d0 0x7f936c197830 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f936400eb10 tx=0x7f936400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:43.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f93697fa640 1 -- 172.21.15.49:0/4082503021 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364004b20 con 0x7f936c1067d0 2024-03-26T18:17:43.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f93697fa640 1 -- 172.21.15.49:0/4082503021 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9364004cc0 con 0x7f936c1067d0 2024-03-26T18:17:43.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f93729b3640 1 -- 172.21.15.49:0/4082503021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c1a6910 con 0x7f936c1067d0 2024-03-26T18:17:43.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f93697fa640 1 -- 172.21.15.49:0/4082503021 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364010760 con 0x7f936c1067d0 2024-03-26T18:17:43.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.282+0000 7f93729b3640 1 -- 172.21.15.49:0/4082503021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c1a6e10 con 0x7f936c1067d0 2024-03-26T18:17:43.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.283+0000 7f93729b3640 1 -- 172.21.15.49:0/4082503021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f936c0686b0 con 0x7f936c1067d0 2024-03-26T18:17:43.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.285+0000 7f93697fa640 1 -- 172.21.15.49:0/4082503021 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f9364010900 con 0x7f936c1067d0 2024-03-26T18:17:43.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.285+0000 7f93697fa640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f93540a3090 0x7f93540a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:17:43.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.285+0000 7f936bfff640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f93540a3090 0x7f93540a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:17:43.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.286+0000 7f93697fa640 1 -- 172.21.15.49:0/4082503021 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f93640d02d0 con 0x7f936c1067d0 2024-03-26T18:17:43.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.286+0000 7f936bfff640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f93540a3090 0x7f93540a5550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f935c009230 tx=0x7f935c005fd0 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:17:43.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.289+0000 7f93697fa640 1 -- 172.21.15.49:0/4082503021 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9364053740 con 0x7f936c1067d0 2024-03-26T18:17:43.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.526+0000 7f93729b3640 1 -- 172.21.15.49:0/4082503021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f936c069010 con 0x7f936c1067d0 2024-03-26T18:17:43.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.553+0000 7f93697fa640 1 -- 172.21.15.49:0/4082503021 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742374 (secure 0 0 0) 0x7f936c069010 con 0x7f936c1067d0 2024-03-26T18:17:43.539 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: "epoch": 26, 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: "active_gid": 15516, 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: "active_name": "x", 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: "active_addrs": { 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6800", 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2879593470 2024-03-26T18:17:43.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6801", 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2879593470 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: "active_addr": "172.21.15.49:6801/2879593470", 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: "active_change": "2024-03-26T18:02:05.919837+0000", 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: "active_mgr_features": 4540701547738038271, 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: "available": true, 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: "standbys": [ 2024-03-26T18:17:43.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "gid": 25412, 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "z", 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_features": 4540701547738038271, 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.542 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.543 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.544 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.545 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.546 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.547 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.548 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.549 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.552 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.553 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.554 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.555 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.556 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.557 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.559 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.560 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.561 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.562 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.563 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.564 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.566 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.567 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.568 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.569 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.570 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.571 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-03-26T18:17:43.572 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.573 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.574 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.575 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.576 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-03-26T18:17:43.577 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.578 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.579 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T18:17:43.584 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T18:17:43.585 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T18:17:43.586 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T18:17:43.587 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.588 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.589 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-03-26T18:17:43.591 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.592 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.594 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.595 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.597 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.598 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.599 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.600 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.601 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.602 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.603 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T18:17:43.604 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T18:17:43.605 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T18:17:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.608 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.609 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "grafana_dashboards_path": { 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "grafana_dashboards_path", 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.610 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.611 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.612 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.613 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.614 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.616 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.617 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.618 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.619 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T18:17:43.620 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T18:17:43.621 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.623 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.624 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.625 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.626 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T18:17:43.627 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T18:17:43.629 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.630 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.631 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.632 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.633 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.634 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.635 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.636 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.637 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.639 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T18:17:43.640 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.642 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.643 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.644 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.645 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.646 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.647 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.649 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.650 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.651 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.652 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.653 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.654 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.655 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.656 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.657 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.658 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.659 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.660 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.661 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.662 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.663 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.665 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.666 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.667 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T18:17:43.669 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T18:17:43.670 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T18:17:43.671 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "{}", 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T18:17:43.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T18:17:43.673 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T18:17:43.674 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T18:17:43.675 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T18:17:43.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.677 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T18:17:43.678 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T18:17:43.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T18:17:43.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.681 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T18:17:43.682 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T18:17:43.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T18:17:43.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T18:17:43.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T18:17:43.686 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T18:17:43.687 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T18:17:43.688 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T18:17:43.689 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T18:17:43.690 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T18:17:43.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T18:17:43.692 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T18:17:43.693 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-03-26T18:17:43.694 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-03-26T18:17:43.695 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.696 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.698 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.699 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.700 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.701 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.702 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.703 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.704 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.705 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.706 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.707 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.708 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.709 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.710 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.711 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.712 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.713 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.714 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.715 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.716 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.717 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.718 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.719 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.720 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-03-26T18:17:43.721 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.722 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.723 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.724 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.725 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:43.726 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-03-26T18:17:43.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.729 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.730 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.731 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.732 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.733 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.734 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.735 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.736 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-03-26T18:17:43.737 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-03-26T18:17:43.738 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-03-26T18:17:43.739 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:43.740 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.741 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.742 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.743 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.744 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.745 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.746 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.749 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.750 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.751 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.753 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.754 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.755 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.756 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.757 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.758 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.759 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.760 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.761 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.762 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.763 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.764 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.765 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.766 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.767 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.768 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.769 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.770 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.772 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.773 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.774 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.776 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.777 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.778 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.779 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.780 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.781 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.782 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.783 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.784 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.785 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.786 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.787 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.789 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.790 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.791 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.792 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.793 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.794 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.795 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.796 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.797 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.798 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.799 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.800 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.801 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.802 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.803 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.804 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.805 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-03-26T18:17:43.806 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.807 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.808 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.809 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.810 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.811 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.812 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.813 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.814 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.815 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.816 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.817 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.818 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.819 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.820 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.821 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.825 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.826 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.827 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.828 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.829 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.830 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.831 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.832 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.833 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.834 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-03-26T18:17:43.835 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-03-26T18:17:43.836 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.837 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.838 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.839 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.840 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.841 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-03-26T18:17:43.842 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T18:17:43.843 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-03-26T18:17:43.844 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-26T18:17:43.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-26T18:17:43.846 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-03-26T18:17:43.847 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-26T18:17:43.848 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.849 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.851 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.852 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.853 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.854 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.855 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.856 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.858 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.859 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.860 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.861 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.861 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-26T18:17:43.861 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-26T18:17:43.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.861 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.861 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.861 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.861 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.862 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.864 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.865 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.866 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.867 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.868 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-03-26T18:17:43.869 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.870 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.871 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.872 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.873 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.875 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.876 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.877 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.878 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.879 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-03-26T18:17:43.880 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.884 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.885 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.886 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.887 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.888 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.889 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.890 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.891 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.892 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.893 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.895 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.896 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.897 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.898 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.899 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.901 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.902 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.903 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.904 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.905 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.906 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.907 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.908 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.909 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.910 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.911 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.912 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.913 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.914 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.915 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.916 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.917 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.918 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.919 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.920 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.921 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.924 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.925 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.926 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.927 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.928 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.929 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.930 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.931 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.933 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:43.934 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-03-26T18:17:43.935 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-03-26T18:17:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-03-26T18:17:43.937 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.938 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.939 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.940 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.941 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.942 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-03-26T18:17:43.943 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-03-26T18:17:43.944 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-03-26T18:17:43.945 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.947 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.948 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.949 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.950 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.951 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.952 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.953 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.954 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.955 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.956 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-03-26T18:17:43.957 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.958 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.959 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_no_wait": { 2024-03-26T18:17:43.960 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.962 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.963 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.964 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.965 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.966 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.967 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.968 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.969 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.970 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.971 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.972 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: "gid": 25415, 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "y", 2024-03-26T18:17:43.973 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_features": 4540701547738038271, 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:43.974 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.975 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.976 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.977 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.978 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.979 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.980 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-03-26T18:17:43.981 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.982 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.983 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-03-26T18:17:43.984 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.985 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.986 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.987 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:43.988 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.991 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.992 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-03-26T18:17:43.993 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-03-26T18:17:43.994 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-03-26T18:17:43.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:43.996 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:43.997 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.998 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:43.999 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.000 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-03-26T18:17:44.001 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-26T18:17:44.002 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-03-26T18:17:44.003 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.004 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.005 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.006 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.007 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-03-26T18:17:44.008 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-03-26T18:17:44.009 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-03-26T18:17:44.010 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.011 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.012 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.013 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.014 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.015 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.016 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.018 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.019 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.020 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.021 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.022 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.023 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.024 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.025 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.026 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.027 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.028 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.029 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.030 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-03-26T18:17:44.032 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-03-26T18:17:44.033 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-03-26T18:17:44.034 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-03-26T18:17:44.035 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.040 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "grafana_dashboards_path": { 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "grafana_dashboards_path", 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.041 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.042 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-03-26T18:17:44.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.045 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.046 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.047 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.048 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T18:17:44.056 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T18:17:44.057 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T18:17:44.058 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T18:17:44.059 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.060 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.061 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.062 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.063 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.064 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.065 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.066 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-03-26T18:17:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.069 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.070 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.071 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.072 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.073 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.074 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.075 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.076 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.077 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T18:17:44.078 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T18:17:44.079 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T18:17:44.080 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-03-26T18:17:44.081 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.082 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T18:17:44.083 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.085 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.086 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.087 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.088 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T18:17:44.089 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T18:17:44.090 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T18:17:44.091 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T18:17:44.092 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T18:17:44.093 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-03-26T18:17:44.094 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T18:17:44.095 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T18:17:44.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T18:17:44.097 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T18:17:44.098 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T18:17:44.099 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T18:17:44.100 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "{}", 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.101 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.102 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.103 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.104 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.105 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.106 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.107 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.108 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.109 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.110 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.111 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T18:17:44.112 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.113 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.114 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.115 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.117 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.118 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.119 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.120 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.121 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.122 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.123 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.124 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.125 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.126 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.127 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.129 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.130 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.131 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.132 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.133 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.134 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.135 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.136 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.137 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.138 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-03-26T18:17:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-26T18:17:44.140 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.143 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.144 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.145 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.148 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.149 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.150 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-03-26T18:17:44.151 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-03-26T18:17:44.152 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.153 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.154 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.155 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.156 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.157 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.158 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.159 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.159 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.159 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.159 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.159 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.159 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.161 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.161 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.162 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.163 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.164 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-03-26T18:17:44.165 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.166 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.167 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.168 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.169 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.170 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-03-26T18:17:44.171 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:44.172 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.173 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.174 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.175 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.176 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.177 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.178 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.179 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.180 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-03-26T18:17:44.181 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.182 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.183 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.184 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-03-26T18:17:44.185 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.186 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.187 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.188 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.189 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.190 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.191 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.192 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.193 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-03-26T18:17:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-03-26T18:17:44.211 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.212 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.212 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.212 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.212 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.212 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.212 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.212 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-03-26T18:17:44.212 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T18:17:44.212 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-03-26T18:17:44.213 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T18:17:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-03-26T18:17:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-03-26T18:17:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-26T18:17:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T18:17:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-26T18:17:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-26T18:17:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-03-26T18:17:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T18:17:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-26T18:17:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-03-26T18:17:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-03-26T18:17:44.325 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.326 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.327 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.328 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.329 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.330 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.331 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.333 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.334 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.335 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.336 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.337 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.337 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.337 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-03-26T18:17:44.337 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-03-26T18:17:44.337 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.337 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.337 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.337 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.337 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.338 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.339 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.340 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-03-26T18:17:44.341 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-03-26T18:17:44.342 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-03-26T18:17:44.343 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.345 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-03-26T18:17:44.345 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-03-26T18:17:44.345 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.345 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.345 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.345 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.345 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.345 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.346 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.346 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.346 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.346 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.346 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.346 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.346 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.347 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-26T18:17:44.347 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.347 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.347 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.347 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-03-26T18:17:44.347 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-03-26T18:17:44.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.347 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.347 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.348 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.349 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.350 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.351 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.352 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.353 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.354 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.355 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.357 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.358 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.359 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.360 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.361 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.362 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.363 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.364 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.364 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.364 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.364 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.366 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.368 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.369 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.370 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.371 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.372 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.373 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.374 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.374 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.374 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.374 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.374 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.374 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.374 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.375 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.376 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.377 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.378 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.378 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T18:17:44.378 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.378 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.378 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.378 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-03-26T18:17:44.378 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-03-26T18:17:44.378 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.378 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.379 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-03-26T18:17:44.380 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-03-26T18:17:44.381 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-03-26T18:17:44.383 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-26T18:17:44.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:44.385 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-03-26T18:17:44.386 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.387 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.388 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.389 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.390 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.391 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.392 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.393 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.394 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.395 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.396 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.397 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-03-26T18:17:44.398 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.399 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.399 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.399 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.399 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.399 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.399 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.399 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.399 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.399 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.399 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.400 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.401 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.402 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.403 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.404 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.405 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.406 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.407 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.408 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-03-26T18:17:44.409 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.410 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_no_wait": { 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.414 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.415 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.416 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.417 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.418 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.419 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.420 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.421 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-03-26T18:17:44.422 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.424 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: "modules": [ 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: "dashboard", 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: "iostat", 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: "nfs", 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: "restful", 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_schedule" 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.425 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.428 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.429 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.430 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.433 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.434 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.435 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.436 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.437 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.438 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.439 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2024-03-26T18:17:44.440 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2024-03-26T18:17:44.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:44.442 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2024-03-26T18:17:44.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2024-03-26T18:17:44.445 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.446 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2024-03-26T18:17:44.447 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.448 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.451 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.452 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.453 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2024-03-26T18:17:44.454 INFO:teuthology.orchestra.run.smithi049.stdout: "read", 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap", 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap-read" 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.456 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2024-03-26T18:17:44.458 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.460 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.461 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.462 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.463 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.464 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.465 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.466 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.467 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.468 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.469 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2024-03-26T18:17:44.470 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.471 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.472 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.473 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2024-03-26T18:17:44.474 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2024-03-26T18:17:44.474 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.474 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.474 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.474 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-26T18:17:44.474 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.474 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.474 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.474 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:44.474 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.475 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.476 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.477 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T18:17:44.478 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T18:17:44.479 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T18:17:44.480 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.481 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.482 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.483 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T18:17:44.484 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.485 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.486 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:44.487 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.488 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.489 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.490 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.491 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2024-03-26T18:17:44.492 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "grafana_dashboards_path": { 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "grafana_dashboards_path", 2024-03-26T18:17:44.493 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2024-03-26T18:17:44.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "hw_monitoring": { 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hw_monitoring", 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.497 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.498 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.499 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.500 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.501 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.502 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.503 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.504 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.504 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.504 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T18:17:44.504 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T18:17:44.504 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.504 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.504 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.504 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T18:17:44.505 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.506 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2024-03-26T18:17:44.507 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2024-03-26T18:17:44.508 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.509 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "oob_default_addr": { 2024-03-26T18:17:44.510 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "oob_default_addr", 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "169.254.1.1", 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2024-03-26T18:17:44.511 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2024-03-26T18:17:44.512 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.513 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.514 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.515 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.516 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.517 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.518 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.519 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.520 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.521 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.522 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2024-03-26T18:17:44.523 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.525 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.526 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.527 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.528 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.529 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.530 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.531 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.532 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T18:17:44.533 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.534 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2024-03-26T18:17:44.535 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.536 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.537 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.539 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T18:17:44.540 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T18:17:44.541 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.542 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.543 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T18:17:44.544 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T18:17:44.545 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.548 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.549 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.550 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.551 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.552 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.553 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.554 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.555 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "{}", 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.556 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.557 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.558 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.559 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.560 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.561 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.562 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.563 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.564 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.566 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T18:17:44.567 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.569 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.569 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.569 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.569 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.569 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.569 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.569 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.570 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.571 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.571 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.571 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.571 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.571 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T18:17:44.571 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T18:17:44.571 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.571 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.571 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T18:17:44.572 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.573 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.573 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.573 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.573 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.573 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.573 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.573 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.573 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.573 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.573 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.574 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T18:17:44.574 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T18:17:44.574 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.574 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.574 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.574 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2024-03-26T18:17:44.574 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.574 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.575 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.575 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.575 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.575 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.575 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.575 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T18:17:44.575 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T18:17:44.575 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T18:17:44.576 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T18:17:44.577 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.578 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.579 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.580 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.581 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.582 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.583 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2024-03-26T18:17:44.584 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.585 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.586 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.587 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.588 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.589 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.590 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.591 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.592 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.593 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.594 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-26T18:17:44.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2024-03-26T18:17:44.597 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.598 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.599 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.600 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.601 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.602 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.602 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-26T18:17:44.602 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.602 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.602 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.603 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2024-03-26T18:17:44.603 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2024-03-26T18:17:44.603 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.603 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.603 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.603 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.604 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.604 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.604 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2024-03-26T18:17:44.604 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.604 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.604 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.604 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.604 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.605 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.606 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.607 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.608 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2024-03-26T18:17:44.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.610 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T18:17:44.614 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.616 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.617 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.618 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.619 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.620 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.621 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2024-03-26T18:17:44.622 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.623 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.624 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.625 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.626 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2024-03-26T18:17:44.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2024-03-26T18:17:44.628 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.630 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.631 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.632 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.633 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.634 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.635 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.636 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.637 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.638 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.639 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.640 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.642 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.643 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.645 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.646 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.647 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.649 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.650 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.651 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.653 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.654 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.655 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.656 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.658 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.659 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.660 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.661 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.663 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.664 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.665 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.666 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2024-03-26T18:17:44.667 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2024-03-26T18:17:44.668 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.669 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2024-03-26T18:17:44.670 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.671 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.672 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.673 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.674 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.675 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.676 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.678 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.679 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.680 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.683 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.684 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.685 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.687 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.688 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.689 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.690 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.691 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.692 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.693 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2024-03-26T18:17:44.694 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.695 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.696 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.697 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.697 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.697 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.697 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.697 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.697 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.697 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.697 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.698 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.699 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.700 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.701 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.701 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.701 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.701 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.701 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.701 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.701 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.701 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.703 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.704 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.705 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.706 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.707 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.708 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.709 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.710 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.711 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.712 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.714 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.715 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.716 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.717 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.718 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.719 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.721 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.722 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.723 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2024-03-26T18:17:44.724 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.725 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.726 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2024-03-26T18:17:44.726 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2024-03-26T18:17:44.726 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.726 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.726 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.726 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.726 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.726 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.726 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.727 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.728 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.729 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.730 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.731 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.732 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.733 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.734 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.735 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.736 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.737 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2024-03-26T18:17:44.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2024-03-26T18:17:44.739 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.740 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.741 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.742 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.743 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.744 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.745 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.746 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.747 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.748 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2024-03-26T18:17:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2024-03-26T18:17:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.750 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2024-03-26T18:17:44.751 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2024-03-26T18:17:44.752 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.753 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.754 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.756 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.757 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.759 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.760 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.761 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.762 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.763 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.764 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.765 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.766 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.767 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.768 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.769 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.770 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.771 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.772 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.772 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2024-03-26T18:17:44.772 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2024-03-26T18:17:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2024-03-26T18:17:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2024-03-26T18:17:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2024-03-26T18:17:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2024-03-26T18:17:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2024-03-26T18:17:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2024-03-26T18:17:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2024-03-26T18:17:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2024-03-26T18:17:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T18:17:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2024-03-26T18:17:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T18:17:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_no_wait": { 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2024-03-26T18:17:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2024-03-26T18:17:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2024-03-26T18:17:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2024-03-26T18:17:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2024-03-26T18:17:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2024-03-26T18:17:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T18:17:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "services": { 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "dashboard": "https://172.21.15.49:8443/" 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "always_on_modules": { 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "octopus": [ 2024-03-26T18:17:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "pacific": [ 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-26T18:17:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "quincy": [ 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-26T18:17:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "reef": [ 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-26T18:17:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "squid": [ 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2024-03-26T18:17:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "last_failure_osd_epoch": 95, 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "active_clients": [ 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3916372512 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "libcephsqlite", 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3963341344 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-26T18:17:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 666726144 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1247564452 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4014986038 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2024-03-26T18:17:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-03-26T18:17:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-03-26T18:17:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-03-26T18:17:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2024-03-26T18:17:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3799018025 2024-03-26T18:17:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-03-26T18:17:44.884 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:17:44.886 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:43 smithi115 ceph-mon[36174]: pgmap v489: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 5.6 MiB/s wr, 601 op/s 2024-03-26T18:17:44.886 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:43 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3237755595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:17:44.886 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:43 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2715147551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:17:44.886 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:43 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4082503021' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T18:17:44.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:43 smithi171 ceph-mon[36033]: pgmap v489: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 5.6 MiB/s wr, 601 op/s 2024-03-26T18:17:44.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:43 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3237755595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:17:44.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:43 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2715147551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:17:44.886 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:43 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4082503021' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T18:17:44.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 -- 172.21.15.49:0/4082503021 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f93540a3090 msgr2=0x7f93540a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:44.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f93540a3090 0x7f93540a5550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f935c009230 tx=0x7f935c005fd0 comp rx=0 tx=0).stop 2024-03-26T18:17:44.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 -- 172.21.15.49:0/4082503021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c1067d0 msgr2=0x7f936c197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:17:44.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c1067d0 0x7f936c197830 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f936400eb10 tx=0x7f936400c2d0 comp rx=0 tx=0).stop 2024-03-26T18:17:44.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 -- 172.21.15.49:0/4082503021 shutdown_connections 2024-03-26T18:17:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f93540a3090 0x7f93540a5550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f936c101180 0x7f936c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f936c1067d0 0x7f936c197830 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 --2- 172.21.15.49:0/4082503021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f936c1007c0 0x7f936c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:17:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 -- 172.21.15.49:0/4082503021 >> 172.21.15.49:0/4082503021 conn(0x7f936c0778c0 msgr2=0x7f936c102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:17:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 -- 172.21.15.49:0/4082503021 shutdown_connections 2024-03-26T18:17:44.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:17:43.559+0000 7f93729b3640 1 -- 172.21.15.49:0/4082503021 wait complete. 2024-03-26T18:17:44.894 DEBUG:tasks.check_counter:Getting stats from a 2024-03-26T18:17:44.894 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-26T18:17:44.894 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:43 smithi049 ceph-mon[32331]: pgmap v489: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 5.6 MiB/s wr, 601 op/s 2024-03-26T18:17:44.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:43 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3237755595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:17:44.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:43 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2715147551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:17:44.895 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:43 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4082503021' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T18:17:45.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:44 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:17:45.210 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:17:45.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:44 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:17:45.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:44 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:17:45.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:45 smithi049 ceph-mon[32331]: pgmap v490: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-03-26T18:17:45.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:45 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:17:45.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:45 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:17:45.934 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:45 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:17:46.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:45 smithi171 ceph-mon[36033]: pgmap v490: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-03-26T18:17:46.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:45 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:17:46.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:45 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:17:46.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:45 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:17:46.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:45 smithi115 ceph-mon[36174]: pgmap v490: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-03-26T18:17:46.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:45 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:17:46.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:45 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:17:46.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:45 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' 2024-03-26T18:17:47.522 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 2108, 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2110, 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 631767, 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 670345, 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 6, 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 2, 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.194803832, 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.057464629, 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.053027191, 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.069296309, 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:47.523 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2108, 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.159852504, 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000075831 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 940, 2024-03-26T18:17:47.524 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1160, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 231720, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 852720, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 8, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 5, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.108972599, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.033163844, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.022287998, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.037582272, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1157, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.069519844, 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000060086 2024-03-26T18:17:47.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 554368 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 343, 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 322, 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1696828, 2024-03-26T18:17:47.526 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 184766, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 2, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.057175085, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.018424925, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.040702480, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.006785204, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 322, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.024178088, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000075087 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.527 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1696560, 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 184448 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 5063, 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.168751133, 2024-03-26T18:17:47.528 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000033330 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.529 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 0, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 0, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 0, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 0, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-03-26T18:17:47.530 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 1, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:17:47.531 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:17:47.532 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 0, 2024-03-26T18:17:47.533 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 0 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 0, 2024-03-26T18:17:47.534 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-03-26T18:17:47.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 0, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 6, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 0, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 2, 2024-03-26T18:17:47.536 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 2, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 4194304, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 4195956, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 4195815, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 6 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 3, 2024-03-26T18:17:47.537 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 3, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 0, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 1, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 1, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 0, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 0, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 0, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 0, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 0, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 0, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 0, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 49976, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 215420 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.538 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 0, 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 0, 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 0, 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 0, 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 0, 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.540 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.541 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-03-26T18:17:47.542 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.543 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.543 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.543 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-03-26T18:17:47.543 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.543 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.544 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.546 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-03-26T18:17:47.547 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-03-26T18:17:47.548 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.549 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.550 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 0, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 2, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 2, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 2, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 0, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 0, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 1229, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:17:47.551 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-26T18:17:47.552 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-26T18:17:47.553 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 190217, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 54, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-26T18:17:47.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 9424, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 181, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 8464, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 13, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-03-26T18:17:47.555 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 3047, 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 21, 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 3, 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 3047, 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3047, 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 6.818813590, 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.002237877 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3047, 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 3048 2024-03-26T18:17:47.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 3047, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 3047, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 0, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 1987, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 0, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 1058, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 0, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 0, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2024-03-26T18:17:47.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 1, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-03-26T18:17:47.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-03-26T18:17:47.559 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 113, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 23, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 5, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 7, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 2, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 0, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 2, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 0 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.560 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 0, 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 0 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 0, 2024-03-26T18:17:47.561 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 0, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 0, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 3387, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 2235192, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 3387, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-26T18:17:47.562 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 3387, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 2235192, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-26T18:17:47.563 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 3047, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 1657, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 3047, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1657, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-03-26T18:17:47.564 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 3047, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 3047, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 3047, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 3047, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:47.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:47.566 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x561ee9bb86e0": { 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-26T18:17:47.567 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-26T18:17:47.568 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-03-26T18:17:47.568 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-03-26T18:17:47.568 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:47.568 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:47.568 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:47.568 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:47.568 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:47.568 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:47.568 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:17:48.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:47 smithi049 ceph-mon[32331]: pgmap v491: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-03-26T18:17:48.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:47 smithi171 ceph-mon[36033]: pgmap v491: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-03-26T18:17:48.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:47 smithi115 ceph-mon[36174]: pgmap v491: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 3.3 MiB/s wr, 350 op/s 2024-03-26T18:17:48.494 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-26T18:17:48.494 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-26T18:17:48.494 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-03-26T18:17:48.494 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-26T18:17:48.494 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T18:17:48.494 DEBUG:tasks.check_counter:Getting stats from d 2024-03-26T18:17:48.495 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-26T18:17:48.753 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:17:49.964 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:49 smithi049 ceph-mon[32331]: pgmap v492: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 2.3 MiB/s wr, 251 op/s 2024-03-26T18:17:50.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:49 smithi171 ceph-mon[36033]: pgmap v492: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 2.3 MiB/s wr, 251 op/s 2024-03-26T18:17:50.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:49 smithi115 ceph-mon[36174]: pgmap v492: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 2.3 MiB/s wr, 251 op/s 2024-03-26T18:17:50.548 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:17:50.548 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:17:50.548 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 4, 2024-03-26T18:17:50.548 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 4, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 3024, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 448, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.001804955, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000045444, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000784778, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000886704, 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000295568 2024-03-26T18:17:50.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 308, 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 297, 2024-03-26T18:17:50.550 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1605856, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 182960, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.058482700, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.018108031, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.038375834, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.010630119, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 296, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.016968259, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000057325 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:50.551 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1605856, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 182960 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 2, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 226, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 320, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 195536, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-03-26T18:17:50.552 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.018168100, 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.012847696, 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.002030409, 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 223, 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.016558609, 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000074253 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.553 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 195536 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:17:50.554 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:17:50.555 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:17:50.556 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 149057, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 32, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-03-26T18:17:50.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-26T18:17:50.558 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 310, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1562367, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 310, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 310, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1562367, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:50.559 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:50.560 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:17:51.354 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-26T18:17:51.355 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-26T18:17:51.355 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-26T18:17:51.355 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-26T18:17:51.355 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-26T18:17:51.355 DEBUG:tasks.check_counter:Getting stats from g 2024-03-26T18:17:51.355 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-26T18:17:51.612 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:17:51.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:51 smithi049 ceph-mon[32331]: pgmap v493: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-03-26T18:17:52.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:51 smithi171 ceph-mon[36033]: pgmap v493: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-03-26T18:17:52.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:51 smithi115 ceph-mon[36174]: pgmap v493: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-03-26T18:17:53.407 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:17:53.407 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:17:53.407 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1014, 2024-03-26T18:17:53.407 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1015, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 341575, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 322135, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 7, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 2, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.129424626, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.046006470, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.032777800, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.039906465, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1013, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.099856909, 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000098575 2024-03-26T18:17:53.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1918, 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2140, 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 490263, 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 1165681, 2024-03-26T18:17:53.409 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 7, 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 5, 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.171679751, 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.049287396, 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.045662586, 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.065571372, 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2137, 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.123826564, 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000057944 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.410 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 556336 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 347, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 326, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1698452, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 186693, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.061380547, 2024-03-26T18:17:53.411 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.018325070, 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.048602782, 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.007132535, 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 326, 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.032441705, 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000099514 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1697104, 2024-03-26T18:17:53.412 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 185728 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 4861, 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.147103928, 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000030262 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.413 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 0, 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 0, 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.414 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 0, 2024-03-26T18:17:53.415 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 1, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2024-03-26T18:17:53.416 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:17:53.417 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:17:53.418 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 0 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 0, 2024-03-26T18:17:53.419 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2024-03-26T18:17:53.420 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 0, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 8, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 0, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2024-03-26T18:17:53.421 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 2, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 2, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 4194304, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 4196655, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 4196514, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.422 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 8 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 3, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 3, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 0, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 1, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 1, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 0, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 0, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 0, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 0, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 0, 2024-03-26T18:17:53.423 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 0, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 0, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 49960, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 215420 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 0, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 0, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 0, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 0, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 0, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.425 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.426 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.427 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2024-03-26T18:17:53.428 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.429 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.430 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.431 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.432 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2024-03-26T18:17:53.433 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2024-03-26T18:17:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 2, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 4, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 2, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 4, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 0, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 0, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 3228, 2024-03-26T18:17:53.436 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:17:53.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:17:53.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:17:53.439 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 198297, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 58, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 9424, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 181, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:17:53.440 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 8464, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 13, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 2932, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 21, 2024-03-26T18:17:53.441 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 3, 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 2932, 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2932, 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 53.530536824, 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.018257345 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2932, 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 2933 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 2932, 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 2932, 2024-03-26T18:17:53.442 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 1909, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 1021, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2024-03-26T18:17:53.443 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 1, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:17:53.444 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2024-03-26T18:17:53.445 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 113, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 23, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 6, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 8, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 2, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 0, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 2, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 0 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.446 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 0, 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 0 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 0, 2024-03-26T18:17:53.447 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed_ops": 0, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 0, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 3275, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 2215143, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 3275, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-26T18:17:53.448 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 3275, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 2215143, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-26T18:17:53.449 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 2932, 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 2356, 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 2932, 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 2356, 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2024-03-26T18:17:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 2932, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 2932, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 2932, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 2932, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-03-26T18:17:53.452 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x55c181c546e0": { 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2024-03-26T18:17:53.453 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-26T18:17:53.454 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-26T18:17:53.454 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2024-03-26T18:17:53.454 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2024-03-26T18:17:53.454 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:53.454 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:53.454 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:53.454 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:53.454 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:53.454 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:53.454 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:17:54.068 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:53 smithi049 ceph-mon[32331]: pgmap v494: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 594 KiB/s wr, 81 op/s 2024-03-26T18:17:54.097 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-26T18:17:54.097 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-26T18:17:54.097 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-03-26T18:17:54.097 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-26T18:17:54.097 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T18:17:54.097 DEBUG:tasks.check_counter:Getting stats from j 2024-03-26T18:17:54.098 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-26T18:17:54.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:53 smithi171 ceph-mon[36033]: pgmap v494: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 594 KiB/s wr, 81 op/s 2024-03-26T18:17:54.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:53 smithi115 ceph-mon[36174]: pgmap v494: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 594 KiB/s wr, 81 op/s 2024-03-26T18:17:54.351 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:17:56.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:55 smithi171 ceph-mon[36033]: pgmap v495: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 229 KiB/s wr, 38 op/s 2024-03-26T18:17:56.116 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:55 smithi049 ceph-mon[32331]: pgmap v495: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 229 KiB/s wr, 38 op/s 2024-03-26T18:17:56.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:55 smithi115 ceph-mon[36174]: pgmap v495: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 229 KiB/s wr, 38 op/s 2024-03-26T18:17:56.248 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-03-26T18:17:56.248 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.001546086, 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000352923, 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T18:17:56.249 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:17:56.250 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:17:56.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.251 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:17:56.251 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 310, 2024-03-26T18:17:56.251 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 299, 2024-03-26T18:17:56.251 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1606400, 2024-03-26T18:17:56.251 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 184240, 2024-03-26T18:17:56.251 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2024-03-26T18:17:56.251 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2024-03-26T18:17:56.251 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.059178144, 2024-03-26T18:17:56.251 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.017847020, 2024-03-26T18:17:56.252 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.043885845, 2024-03-26T18:17:56.252 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.006939999, 2024-03-26T18:17:56.252 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:56.252 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 298, 2024-03-26T18:17:56.252 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.024567903, 2024-03-26T18:17:56.252 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000082442 2024-03-26T18:17:56.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.252 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:56.252 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:56.253 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:56.253 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:56.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.253 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 1606400, 2024-03-26T18:17:56.253 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 184240 2024-03-26T18:17:56.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.253 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:17:56.253 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 7, 2024-03-26T18:17:56.253 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 231, 2024-03-26T18:17:56.253 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 3504, 2024-03-26T18:17:56.254 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 274656, 2024-03-26T18:17:56.254 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 6, 2024-03-26T18:17:56.254 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2024-03-26T18:17:56.254 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.018828223, 2024-03-26T18:17:56.254 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.012979128, 2024-03-26T18:17:56.254 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.002718495, 2024-03-26T18:17:56.254 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T18:17:56.254 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:56.255 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 229, 2024-03-26T18:17:56.255 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.020151584, 2024-03-26T18:17:56.255 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000087998 2024-03-26T18:17:56.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.255 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:56.255 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:56.255 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:56.255 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:56.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.256 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-03-26T18:17:56.256 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 274656 2024-03-26T18:17:56.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.256 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2024-03-26T18:17:56.256 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2024-03-26T18:17:56.256 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2024-03-26T18:17:56.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.256 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2024-03-26T18:17:56.256 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:17:56.257 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:17:56.258 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:17:56.258 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:17:56.258 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2024-03-26T18:17:56.258 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:17:56.258 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2024-03-26T18:17:56.258 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:17:56.258 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:17:56.258 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:17:56.258 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:17:56.258 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:17:56.259 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 148714, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 30, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2024-03-26T18:17:56.260 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2024-03-26T18:17:56.261 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2024-03-26T18:17:56.261 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:17:56.261 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:17:56.261 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2024-03-26T18:17:56.261 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2024-03-26T18:17:56.261 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2024-03-26T18:17:56.261 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2024-03-26T18:17:56.261 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2024-03-26T18:17:56.261 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2024-03-26T18:17:56.261 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 313, 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 1562654, 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 313, 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2024-03-26T18:17:56.262 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2024-03-26T18:17:56.263 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 313, 2024-03-26T18:17:56.263 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1562654, 2024-03-26T18:17:56.263 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2024-03-26T18:17:56.263 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2024-03-26T18:17:56.263 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2024-03-26T18:17:56.263 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2024-03-26T18:17:56.263 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:56.263 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-03-26T18:17:56.263 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-03-26T18:17:57.146 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-26T18:17:57.146 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-26T18:17:57.146 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-26T18:17:57.146 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-26T18:17:57.146 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-26T18:17:57.146 DEBUG:tasks.check_counter:Getting stats from b 2024-03-26T18:17:57.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-26T18:17:57.551 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:17:57.945 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:57 smithi115 ceph-mon[36174]: pgmap v496: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-03-26T18:17:57.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:57 smithi049 ceph-mon[32331]: pgmap v496: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-03-26T18:17:58.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:57 smithi171 ceph-mon[36033]: pgmap v496: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-03-26T18:17:59.818 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-26T18:17:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:17:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 3232, 2024-03-26T18:17:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 4849, 2024-03-26T18:17:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1043199, 2024-03-26T18:17:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 7371333, 2024-03-26T18:17:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 20, 2024-03-26T18:17:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-03-26T18:17:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.281602133, 2024-03-26T18:17:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.091249582, 2024-03-26T18:17:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.116882549, 2024-03-26T18:17:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.071122554, 2024-03-26T18:17:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4845, 2024-03-26T18:17:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.291576987, 2024-03-26T18:17:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000266579 2024-03-26T18:17:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6279, 2024-03-26T18:17:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001008954, 2024-03-26T18:17:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000160 2024-03-26T18:17:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 49280, 2024-03-26T18:17:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 463136 2024-03-26T18:17:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:17:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2579, 2024-03-26T18:17:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1841, 2024-03-26T18:17:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 2733010, 2024-03-26T18:17:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 1322226, 2024-03-26T18:17:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2024-03-26T18:17:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2024-03-26T18:17:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.197388904, 2024-03-26T18:17:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.056038012, 2024-03-26T18:17:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.123955233, 2024-03-26T18:17:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.040733571, 2024-03-26T18:17:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1843, 2024-03-26T18:17:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.127458903, 2024-03-26T18:17:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000611751 2024-03-26T18:17:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2939, 2024-03-26T18:17:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000525824, 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000178 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1787584, 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 198912 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 3424, 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1782, 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1708044, 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 1266608, 2024-03-26T18:17:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 15, 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 9, 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.355331316, 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.092165342, 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.135625884, 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.124191027, 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1779, 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.313292258, 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000176105 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:17:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4915, 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000916075, 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000186 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 560672, 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 284240 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 622, 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.806822527, 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001297142 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 25, 2024-03-26T18:17:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.007456207, 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000298248 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 1562, 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 1432, 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1432, 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.844916912, 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001986673 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-03-26T18:17:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 130, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 39, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 3, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 48, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 14, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 7, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 27, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 27, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 858, 2024-03-26T18:17:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 3, 2024-03-26T18:17:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 3, 2024-03-26T18:17:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 3, 2024-03-26T18:17:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 1646, 2024-03-26T18:17:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 1160, 2024-03-26T18:17:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 130, 2024-03-26T18:17:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-03-26T18:17:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:17:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 1, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 35, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 878, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 1020, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 1571192001, 2024-03-26T18:17:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 2, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 44, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 846, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:17:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 339, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 285, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 1, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 60 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 37, 2024-03-26T18:17:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-03-26T18:17:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-03-26T18:17:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:17:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 9, 2024-03-26T18:17:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 8, 2024-03-26T18:17:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 1, 2024-03-26T18:17:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-03-26T18:17:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-03-26T18:17:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 35, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 3, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-03-26T18:17:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 1171, 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 1171, 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 4, 2024-03-26T18:17:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-03-26T18:17:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-03-26T18:17:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 4, 2024-03-26T18:17:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 2, 2024-03-26T18:17:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-03-26T18:17:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-03-26T18:17:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-03-26T18:17:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 8924570, 2024-03-26T18:17:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-03-26T18:17:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-03-26T18:17:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 275, 2024-03-26T18:17:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.824343572, 2024-03-26T18:17:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.010270340 2024-03-26T18:17:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-03-26T18:17:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-03-26T18:17:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 48, 2024-03-26T18:17:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 862, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 814, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 67, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 85, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 18, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 48, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 1120, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 1072, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 3, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 856, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 853, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 61084, 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 231804 2024-03-26T18:17:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 1646, 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 40, 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 1562, 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 71, 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 80, 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 816, 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.051565782, 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002514173 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-03-26T18:17:59.840 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 166, 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.042114112, 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000253699 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001523616, 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001523616 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.841 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3, 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000802795, 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000267598 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.843 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.844 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.844 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-03-26T18:17:59.844 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 15, 2024-03-26T18:17:59.844 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.003298989, 2024-03-26T18:17:59.844 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000219932 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 33, 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.037236949, 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001128392 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.845 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 15, 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.213934295, 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.014262286 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 335, 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.065018618, 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000194085 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 13, 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.071784249, 2024-03-26T18:17:59.846 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.005521865 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 9, 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.136238460, 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.015137606 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-03-26T18:17:59.847 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 9, 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.127473253, 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.014163694 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.848 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001854318, 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000927159 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-03-26T18:17:59.850 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 9, 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.021931702, 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002436855 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6, 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.070139774, 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.011689962 2024-03-26T18:17:59.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 7, 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 5, 2024-03-26T18:17:59.852 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 27, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 13, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 926, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:17:59.853 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-03-26T18:17:59.854 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-03-26T18:17:59.855 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 372236, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 177, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 440, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 5, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-03-26T18:17:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9424, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 181, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 307698, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 2647, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-03-26T18:17:59.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 940, 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 5106615, 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 37, 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 940, 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 940, 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 21.209583610, 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.022563386 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.858 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 940, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1004 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 940, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 1, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 939, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 11, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1, 2024-03-26T18:17:59.859 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 276, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 60, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 3, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 529, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 0, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 22, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-03-26T18:17:59.860 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-03-26T18:17:59.861 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 113, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-03-26T18:17:59.862 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 28, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 92, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 0, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 10 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.863 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 10, 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 5, 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 16, 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 8, 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:17:59.864 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 9231, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 4562279, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 9231, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 9231, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 4562279, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 940, 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 5107487, 2024-03-26T18:17:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 940, 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 5107487, 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:17:59.867 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 940, 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 940, 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 940, 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 940, 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:17:59.868 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 8, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 872, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 8, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 1, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 872, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.869 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55f4f2aae500": { 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 1171, 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 4730266, 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 1171, 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:17:59.870 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:17:59.871 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 275, 2024-03-26T18:17:59.871 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 4730266, 2024-03-26T18:17:59.871 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:17:59.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:17:59.871 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:17:59.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:17:59.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:17:59.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:17:59.871 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-26T18:18:00.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:17:59 smithi171 ceph-mon[36033]: pgmap v497: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-03-26T18:18:00.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:17:59 smithi115 ceph-mon[36174]: pgmap v497: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-03-26T18:18:00.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:17:59 smithi049 ceph-mon[32331]: pgmap v497: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-03-26T18:18:00.816 INFO:tasks.check_counter:Daemon mds.b mds.exported=35 2024-03-26T18:18:00.816 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-26T18:18:00.816 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-26T18:18:00.816 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=15 2024-03-26T18:18:00.816 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=9 2024-03-26T18:18:00.816 DEBUG:tasks.check_counter:Getting stats from e 2024-03-26T18:18:00.816 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-26T18:18:01.088 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:18:01.974 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:01 smithi115 ceph-mon[36174]: pgmap v498: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-03-26T18:18:02.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:01 smithi171 ceph-mon[36033]: pgmap v498: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-03-26T18:18:02.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:01 smithi049 ceph-mon[32331]: pgmap v498: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-03-26T18:18:03.748 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-26T18:18:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:18:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2181, 2024-03-26T18:18:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 2183, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 4843001, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 693568, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 7, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 2, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.190183695, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.060748202, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.048314009, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.066272805, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2181, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.128331935, 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000058840 2024-03-26T18:18:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 103, 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 329, 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 25838, 2024-03-26T18:18:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 592456, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.020968913, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.009884759, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.004817345, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.003533440, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 326, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.024414404, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000074890 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 560272 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1444, 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1418, 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 2538761, 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 536035, 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 3, 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 2, 2024-03-26T18:18:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.144177462, 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.036782498, 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.061795377, 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.046239122, 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1418, 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.072966559, 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000051457 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1724592, 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 190368 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5559, 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.168566843, 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000030323 2024-03-26T18:18:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-03-26T18:18:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-03-26T18:18:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-03-26T18:18:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-03-26T18:18:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 23, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 0, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 23, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 23, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 3, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2024-03-26T18:18:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-03-26T18:18:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 1020, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 1571273921, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:18:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-03-26T18:18:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:18:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-03-26T18:18:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 1177, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 4, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 2, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-03-26T18:18:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 8933368, 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 8929205, 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 1177 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 15, 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 853, 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 838, 2024-03-26T18:18:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 10, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 60, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 50, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 23, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 859, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 836, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 50952, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 215420 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2024-03-26T18:18:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-03-26T18:18:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-03-26T18:18:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-03-26T18:18:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-03-26T18:18:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-03-26T18:18:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-03-26T18:18:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 7, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 5, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 5, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 4679, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:18:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-03-26T18:18:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-03-26T18:18:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 190769, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 69, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-03-26T18:18:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9424, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 181, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 125218, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 673, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-03-26T18:18:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 3369, 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 26, 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 4, 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 3369, 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3369, 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 9.860811168, 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002926925 2024-03-26T18:18:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3369, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3370 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 3369, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 3369, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 2134, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2024-03-26T18:18:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1232, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 2, 2024-03-26T18:18:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-03-26T18:18:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 113, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 23, 2024-03-26T18:18:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 5, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 8, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 3, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 3724, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 7051406, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 3724, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3724, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 7051406, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3369, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 4739074, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3369, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 4739074, 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 3369, 2024-03-26T18:18:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 3369, 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3369, 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 3369, 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:18:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55c73ebb06e0": { 2024-03-26T18:18:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:03.797 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-26T18:18:04.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:03 smithi171 ceph-mon[36033]: pgmap v499: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 60 KiB/s wr, 11 op/s 2024-03-26T18:18:04.126 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:03 smithi115 ceph-mon[36174]: pgmap v499: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 60 KiB/s wr, 11 op/s 2024-03-26T18:18:04.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:03 smithi049 ceph-mon[32331]: pgmap v499: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 60 KiB/s wr, 11 op/s 2024-03-26T18:18:04.684 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-26T18:18:04.685 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-26T18:18:04.685 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-03-26T18:18:04.685 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-26T18:18:04.685 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T18:18:04.685 DEBUG:tasks.check_counter:Getting stats from h 2024-03-26T18:18:04.685 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-26T18:18:04.948 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:18:06.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:05 smithi171 ceph-mon[36033]: pgmap v500: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 121 KiB/s wr, 33 op/s 2024-03-26T18:18:06.119 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:05 smithi115 ceph-mon[36174]: pgmap v500: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 121 KiB/s wr, 33 op/s 2024-03-26T18:18:06.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:05 smithi049 ceph-mon[32331]: pgmap v500: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 121 KiB/s wr, 33 op/s 2024-03-26T18:18:06.842 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-26T18:18:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:18:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1145, 2024-03-26T18:18:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1173, 2024-03-26T18:18:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1962955, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 1673865, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 17, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.095185376, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.032270416, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.221452807, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.008318531, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1168, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.153145294, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000131117 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1336, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000398626, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000298 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1736608, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 195280 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1230, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1209, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 693448, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 1176151, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 10, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-03-26T18:18:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.113350530, 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.054941680, 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.065010109, 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.005073005, 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1211, 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.126862248, 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000104758 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2148, 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000457156, 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000212 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 561712 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 49, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 79, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 12313, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 53484, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.007999733, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.002144438, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.004652803, 2024-03-26T18:18:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000835943, 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 77, 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.013155352, 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000170848 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 67, 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000012626, 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000188 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 4384, 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 12960 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 102, 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.118585230, 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001162600 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-03-26T18:18:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000005711, 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000002855 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 168, 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 167, 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 167, 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.082146488, 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000491895 2024-03-26T18:18:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 1, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 11, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 313, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 0, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 313, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 313, 2024-03-26T18:18:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 0, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 292, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 292, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 12, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 320, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 307, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 11, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-03-26T18:18:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 9, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 292, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 4, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 234, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 0, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:18:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 2, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 535, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 242, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 1, 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 24 2024-03-26T18:18:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 5, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 8, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-03-26T18:18:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 579, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 579, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 2, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 2, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 2, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-03-26T18:18:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 5071350, 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 26, 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.559394415, 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.021515169 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 314, 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 314, 2024-03-26T18:18:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 0, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 28, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 29, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 1, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 313, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 313, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 0, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 292, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 295, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 3, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 55460, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 223612 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-03-26T18:18:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 170, 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 168, 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 37, 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000377903, 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000377903 2024-03-26T18:18:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 141, 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.044410740, 2024-03-26T18:18:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000314969 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-03-26T18:18:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 25, 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.037357845, 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001494313 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-03-26T18:18:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-03-26T18:18:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-03-26T18:18:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-03-26T18:18:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 5, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 3, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 8, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 4, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 928, 2024-03-26T18:18:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:18:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:18:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 267866, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 390, 2024-03-26T18:18:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9424, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 181, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:18:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 1111765, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 12256, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 76, 2024-03-26T18:18:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 910621, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 28, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 76, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 76, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1.113978537, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.014657612 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 76, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 106 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 76, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 0, 2024-03-26T18:18:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 76, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 10, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 0, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 26, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 13, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 6, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-03-26T18:18:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 20, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-03-26T18:18:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-03-26T18:18:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 113, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 23, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 9, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 13, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 4, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 28, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 0, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 3 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-03-26T18:18:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-03-26T18:18:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 2420, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 2427479, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 2420, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 2420, 2024-03-26T18:18:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 2427479, 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:18:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 76, 2024-03-26T18:18:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 910621, 2024-03-26T18:18:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 76, 2024-03-26T18:18:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 910621, 2024-03-26T18:18:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 76, 2024-03-26T18:18:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 76, 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 76, 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 76, 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55d87eabc500": { 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 579, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 877046, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 579, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 26, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 877046, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:06.892 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-26T18:18:07.658 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-26T18:18:07.659 INFO:tasks.check_counter:Daemon mds.h mds.imported=9 2024-03-26T18:18:07.659 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-03-26T18:18:07.659 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-26T18:18:07.659 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T18:18:07.659 DEBUG:tasks.check_counter:Getting stats from k 2024-03-26T18:18:07.660 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-26T18:18:07.922 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:18:07.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:07 smithi115 ceph-mon[36174]: pgmap v501: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 108 KiB/s wr, 32 op/s 2024-03-26T18:18:08.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:07 smithi171 ceph-mon[36033]: pgmap v501: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 108 KiB/s wr, 32 op/s 2024-03-26T18:18:08.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:07 smithi049 ceph-mon[32331]: pgmap v501: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 108 KiB/s wr, 32 op/s 2024-03-26T18:18:10.056 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:09 smithi049 ceph-mon[32331]: pgmap v502: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 108 KiB/s wr, 30 op/s 2024-03-26T18:18:10.056 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:09 smithi049 ceph-mon[32331]: osdmap e114: 12 total, 12 up, 12 in 2024-03-26T18:18:10.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:09 smithi171 ceph-mon[36033]: pgmap v502: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 108 KiB/s wr, 30 op/s 2024-03-26T18:18:10.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:09 smithi171 ceph-mon[36033]: osdmap e114: 12 total, 12 up, 12 in 2024-03-26T18:18:10.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:09 smithi115 ceph-mon[36174]: pgmap v502: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 108 KiB/s wr, 30 op/s 2024-03-26T18:18:10.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:09 smithi115 ceph-mon[36174]: osdmap e114: 12 total, 12 up, 12 in 2024-03-26T18:18:11.420 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-03-26T18:18:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:18:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 384, 2024-03-26T18:18:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 671, 2024-03-26T18:18:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 135708, 2024-03-26T18:18:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 155896, 2024-03-26T18:18:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 8, 2024-03-26T18:18:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 1, 2024-03-26T18:18:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.051825911, 2024-03-26T18:18:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.024219617, 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.019437822, 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.005111282, 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 673, 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.061473761, 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000091342 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 982, 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000230961, 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000235 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-26T18:18:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 19, 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 249, 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 4699, 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 656663, 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 13, 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.014471420, 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.007818484, 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.004459825, 2024-03-26T18:18:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.000179013, 2024-03-26T18:18:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 246, 2024-03-26T18:18:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.050573071, 2024-03-26T18:18:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000205581 2024-03-26T18:18:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 8, 2024-03-26T18:18:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000001455, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000181 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1408, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 650992 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 403, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 385, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 1728276, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 255700, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 13, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.062796382, 2024-03-26T18:18:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.017189928, 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.056374105, 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.007140317, 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 381, 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.043024992, 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000112926 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 19, 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000002434, 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000128 2024-03-26T18:18:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1726096, 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 200176 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 52, 2024-03-26T18:18:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001538982, 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000029595 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000006771, 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000003385 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 3, 2024-03-26T18:18:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 3, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001326422, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000442140 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 11, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 11, 2024-03-26T18:18:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 0, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 11, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 10, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 1, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 2, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 8, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 4, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 2, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2024-03-26T18:18:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 4, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 234, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 0, 2024-03-26T18:18:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:18:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 3, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:18:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2024-03-26T18:18:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 6, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 6, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 2, 2024-03-26T18:18:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 2, 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 2, 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 4194304, 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 4195956, 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5, 2024-03-26T18:18:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.008518108, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001703621 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 14, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 14, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 0, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 13, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 14, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 1, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 11, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 12, 2024-03-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 1, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 2, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 2, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 50192, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 223612 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 5, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 3, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 8, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2024-03-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000294261, 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000294261 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001032161, 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000516080 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2024-03-26T18:18:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2024-03-26T18:18:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 0, 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 2, 2024-03-26T18:18:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 2, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 0, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 2, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 1, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 88, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:18:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:18:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2024-03-26T18:18:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 239159, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 74, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:18:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 9664, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 187, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 56320, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 224, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2024-03-26T18:18:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 43, 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 12099, 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 33, 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 43, 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 43, 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.230995421, 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.005371986 2024-03-26T18:18:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 43, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 67 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 43, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 0, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 43, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 10, 2024-03-26T18:18:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 0, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 5, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 7, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 6, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 0, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 20, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:18:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2024-03-26T18:18:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 114, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 24, 2024-03-26T18:18:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 8, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 14, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 6, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 17, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 0, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 2 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 802, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 1773410, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 802, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 802, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 1773410, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 43, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 12099, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 43, 2024-03-26T18:18:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 12099, 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:18:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 43, 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 43, 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 43, 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 43, 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55593985e500": { 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 6, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 1652, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 6, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 5, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 1652, 2024-03-26T18:18:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2024-03-26T18:18:11.468 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2024-03-26T18:18:11.468 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2024-03-26T18:18:11.468 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2024-03-26T18:18:11.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:11.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-03-26T18:18:11.468 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-03-26T18:18:12.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:11 smithi171 ceph-mon[36033]: pgmap v504: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 82 KiB/s wr, 36 op/s 2024-03-26T18:18:12.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:11 smithi115 ceph-mon[36174]: pgmap v504: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 82 KiB/s wr, 36 op/s 2024-03-26T18:18:12.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:11 smithi049 ceph-mon[32331]: pgmap v504: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 82 KiB/s wr, 36 op/s 2024-03-26T18:18:12.355 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-26T18:18:12.356 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-26T18:18:12.356 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-03-26T18:18:12.356 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-26T18:18:12.356 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T18:18:12.356 DEBUG:tasks.check_counter:Getting stats from c 2024-03-26T18:18:12.357 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-26T18:18:12.938 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:18:13.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:13 smithi115 ceph-mon[36174]: pgmap v505: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 KiB/s wr, 35 op/s 2024-03-26T18:18:14.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:13 smithi171 ceph-mon[36033]: pgmap v505: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 KiB/s wr, 35 op/s 2024-03-26T18:18:14.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:13 smithi049 ceph-mon[32331]: pgmap v505: 129 pgs: 4 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 113 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 KiB/s wr, 35 op/s 2024-03-26T18:18:15.084 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-03-26T18:18:15.084 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:18:15.084 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 3183, 2024-03-26T18:18:15.084 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 3183, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1777251, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 1011377, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 2, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.295690831, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.063259837, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.065985646, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.152061351, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3182, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.152843768, 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000048033 2024-03-26T18:18:15.085 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 5, 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 232, 2024-03-26T18:18:15.086 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1036, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 651978, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 8, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.012615633, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.007935569, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.002523595, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000081959, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 229, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.029436206, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000128542 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.087 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 651280 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 355, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 331, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1723260, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 190894, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 2, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.052615130, 2024-03-26T18:18:15.088 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.015780470, 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.040644111, 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.006895638, 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 330, 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.018999919, 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000057575 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1722992, 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 190576 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5285, 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.129268210, 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000024459 2024-03-26T18:18:15.090 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-03-26T18:18:15.091 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 303, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 0, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 303, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-03-26T18:18:15.092 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 303, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 11, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-03-26T18:18:15.093 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:18:15.094 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-03-26T18:18:15.095 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-03-26T18:18:15.096 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.097 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 580, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 2, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 2, 2024-03-26T18:18:15.098 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 5071924, 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 5071350, 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 580 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 304, 2024-03-26T18:18:15.099 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 304, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 0, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 18, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 18, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 0, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 303, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 303, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 0, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 51432, 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 215420 2024-03-26T18:18:15.100 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.101 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.102 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.103 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.104 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.105 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-03-26T18:18:15.105 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.105 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.105 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.105 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.105 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.106 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-03-26T18:18:15.107 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.108 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-03-26T18:18:15.109 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.110 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.111 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:15.112 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 5, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 3, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 4, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 3927, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-03-26T18:18:15.113 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:18:15.114 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:18:15.115 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 193042, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 346, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:18:15.116 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 9664, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 187, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 938583, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 8568, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-03-26T18:18:15.117 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 3182, 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 21, 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 3, 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 3182, 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3182, 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 20.700606301, 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.006505533 2024-03-26T18:18:15.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3182, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3183 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 3182, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 3182, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 2084, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1096, 2024-03-26T18:18:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 1, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-03-26T18:18:15.120 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-03-26T18:18:15.121 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 114, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 24, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 4, 2024-03-26T18:18:15.122 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 6, 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 2, 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-03-26T18:18:15.123 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-26T18:18:15.124 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 3539, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 3161904, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 3539, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3539, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 3161904, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:15.125 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 3182, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 877625, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3182, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 877625, 2024-03-26T18:18:15.126 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 3182, 2024-03-26T18:18:15.127 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 3182, 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3182, 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 3182, 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:15.128 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x5623fa68c6e0": { 2024-03-26T18:18:15.129 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:15.130 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:15.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:15.131 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:15.131 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-03-26T18:18:16.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:15 smithi171 ceph-mon[36033]: pgmap v506: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-03-26T18:18:16.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:15 smithi115 ceph-mon[36174]: pgmap v506: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-03-26T18:18:16.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:15 smithi049 ceph-mon[32331]: pgmap v506: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-03-26T18:18:16.747 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-26T18:18:16.747 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-26T18:18:16.748 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-03-26T18:18:16.748 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-26T18:18:16.748 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T18:18:16.748 DEBUG:tasks.check_counter:Getting stats from f 2024-03-26T18:18:16.748 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-26T18:18:17.017 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:18:18.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:17 smithi171 ceph-mon[36033]: pgmap v507: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-03-26T18:18:18.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:17 smithi171 ceph-mon[36033]: osdmap e115: 12 total, 12 up, 12 in 2024-03-26T18:18:18.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:17 smithi115 ceph-mon[36174]: pgmap v507: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-03-26T18:18:18.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:17 smithi115 ceph-mon[36174]: osdmap e115: 12 total, 12 up, 12 in 2024-03-26T18:18:18.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:17 smithi049 ceph-mon[32331]: pgmap v507: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-03-26T18:18:18.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:17 smithi049 ceph-mon[32331]: osdmap e115: 12 total, 12 up, 12 in 2024-03-26T18:18:19.625 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-03-26T18:18:19.625 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:18:19.625 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 634, 2024-03-26T18:18:19.625 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 610, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 5989594, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 280124, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 7, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 2, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.084645187, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.021511035, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.053712780, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.017048255, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 609, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.043386618, 2024-03-26T18:18:19.626 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000071242 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1724048, 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 192032 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1183, 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1412, 2024-03-26T18:18:19.627 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 19610401, 2024-03-26T18:18:19.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 941211, 2024-03-26T18:18:19.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 8, 2024-03-26T18:18:19.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2024-03-26T18:18:19.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.119757555, 2024-03-26T18:18:19.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.036233816, 2024-03-26T18:18:19.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.038275555, 2024-03-26T18:18:19.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.035416851, 2024-03-26T18:18:19.628 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:19.628 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1409, 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.096193738, 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000068270 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 565648 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 2006, 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2006, 2024-03-26T18:18:19.629 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 17433888, 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 637142, 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.185047987, 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.045358938, 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.061158905, 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.065944360, 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2005, 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.111759885, 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000055740 2024-03-26T18:18:19.630 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-03-26T18:18:19.631 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5845, 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.148540749, 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000025413 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.632 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-03-26T18:18:19.633 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 2083, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 29, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 2054, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 2042, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 12, 2024-03-26T18:18:19.634 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-03-26T18:18:19.635 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:18:19.636 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-03-26T18:18:19.637 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:18:19.638 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-03-26T18:18:19.639 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 5816, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 12, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 2, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 44470127, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 44469553, 2024-03-26T18:18:19.640 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 5816 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 1455, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 1695, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 240, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 80, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 84, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 4, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 2083, 2024-03-26T18:18:19.641 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 2088, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 5, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 60464, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 215420 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-03-26T18:18:19.642 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-03-26T18:18:19.643 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.644 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.645 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.646 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.647 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-03-26T18:18:19.647 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.647 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.647 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.647 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.647 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-03-26T18:18:19.647 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-03-26T18:18:19.648 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-03-26T18:18:19.649 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.650 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.651 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-03-26T18:18:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.653 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 6, 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 4, 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 5, 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-03-26T18:18:19.654 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 4669, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:18:19.655 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:18:19.656 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:18:19.657 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 278880, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 1770, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 9664, 2024-03-26T18:18:19.658 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 187, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 5254836, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 96855, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-03-26T18:18:19.659 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 3460, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 21, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 3, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 3460, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3460, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 27.235481918, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.007871526 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3460, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3461 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.660 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 3460, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 3460, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 2071, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1387, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-03-26T18:18:19.661 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 1, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-03-26T18:18:19.662 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-03-26T18:18:19.663 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 115, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 25, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 8, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 9, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 1, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.664 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-03-26T18:18:19.665 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 3819, 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 42668933, 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 3819, 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:19.666 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3819, 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 42668933, 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:19.667 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 3460, 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 40340111, 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3460, 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 40340111, 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.668 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 3460, 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 3460, 2024-03-26T18:18:19.669 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 3460, 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 3460, 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:19.670 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x556e7072a6e0": { 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:19.671 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:19.672 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:19.673 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-03-26T18:18:19.880 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:19 smithi171 ceph-mon[36033]: pgmap v509: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-03-26T18:18:20.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:19 smithi115 ceph-mon[36174]: pgmap v509: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-03-26T18:18:20.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:19 smithi049 ceph-mon[32331]: pgmap v509: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-03-26T18:18:20.539 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-26T18:18:20.539 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-26T18:18:20.539 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-26T18:18:20.539 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-26T18:18:20.539 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T18:18:20.539 DEBUG:tasks.check_counter:Getting stats from i 2024-03-26T18:18:20.539 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-26T18:18:20.829 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:18:21.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:21 smithi049 ceph-mon[32331]: pgmap v510: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-03-26T18:18:22.081 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:21 smithi171 ceph-mon[36033]: pgmap v510: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-03-26T18:18:22.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:21 smithi115 ceph-mon[36174]: pgmap v510: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-03-26T18:18:22.499 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-03-26T18:18:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:18:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 9699, 2024-03-26T18:18:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 12219, 2024-03-26T18:18:22.499 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 5035372, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 17782304, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 10, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.658693177, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.256892863, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 1.023524878, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.038487026, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 12218, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.455360270, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000119116 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:22.500 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 17604, 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.003120542, 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000177 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1732640, 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 204912 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1423, 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1972, 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 402667, 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 18252769, 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 12, 2024-03-26T18:18:22.501 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 7, 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.188053011, 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.081971833, 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.043453757, 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.046783616, 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1969, 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.277994107, 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000141185 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 612, 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000121842, 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000199 2024-03-26T18:18:22.502 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1408, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 655664 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 874, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 876, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 246488, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 9188339, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 15, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.087300517, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.033430711, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.020505398, 2024-03-26T18:18:22.503 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.027132936, 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 875, 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.147830966, 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000168949 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 33, 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000004112, 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000124 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 6816, 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 3984 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.504 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3327, 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3.199328876, 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000961625 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-03-26T18:18:22.505 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 882, 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.050062341, 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000056760 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 2780, 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 2772, 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2772, 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 7.466148148, 2024-03-26T18:18:22.506 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002693415 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 8, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 29, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 2, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 1789, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 97, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 1692, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 1650, 2024-03-26T18:18:22.507 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 1, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 756, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 756, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 21, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 5379, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 4598, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 1, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 28, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 264, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-03-26T18:18:22.508 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 1, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 1, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 27, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 587, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 4, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 234, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:18:22.509 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 1, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 659, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 2198, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 1, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 4056, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 1976, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 184, 2024-03-26T18:18:22.510 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 1007 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 36, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 1, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 1, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 397, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 240, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 157, 2024-03-26T18:18:22.511 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 1, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 2, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-03-26T18:18:22.512 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 5816, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 5816, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 12, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 12, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 2, 2024-03-26T18:18:22.513 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 44470127, 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 0, 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1816, 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 42.426020150, 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.023362345 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 0 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 1455, 2024-03-26T18:18:22.514 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 1695, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 240, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 80, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 85, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 5, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 1789, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 2114, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 325, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 756, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 1324, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 568, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 68320, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 223612 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-03-26T18:18:22.515 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 3543, 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 2780, 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 365, 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 40, 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 452, 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.205063801, 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002666070 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 36, 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.120705431, 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.003352928 2024-03-26T18:18:22.516 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 157, 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.632352461, 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.004027722 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1207, 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.631265946, 2024-03-26T18:18:22.517 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001351504 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1, 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000131703, 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000131703 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.518 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-03-26T18:18:22.519 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-03-26T18:18:22.520 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4, 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.078775073, 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.019693768 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.521 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 237, 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.259877033, 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001096527 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 114, 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.188418416, 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001652793 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 377, 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2.814571863, 2024-03-26T18:18:22.522 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.007465707 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2, 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.015713059, 2024-03-26T18:18:22.523 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.007856529 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1, 2024-03-26T18:18:22.524 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.003992121, 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.003992121 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-03-26T18:18:22.525 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.526 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 184, 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.515281241, 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002800441 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-03-26T18:18:22.527 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 6, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 4, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 2, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 930, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 465, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 944, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:18:22.528 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:18:22.529 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-03-26T18:18:22.530 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 836477, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 1085, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 968, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 11, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:18:22.531 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 9424, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 181, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 5293302, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 96944, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-03-26T18:18:22.532 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 2983, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 40780964, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 30, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 2983, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2983, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 60.171848423, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.020171588 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2983, 2024-03-26T18:18:22.533 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3321 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 2983, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 103, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 2880, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 0, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 10, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 103, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 1919, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 152, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-03-26T18:18:22.534 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 1, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 2, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 514, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 0, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 20, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-03-26T18:18:22.535 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 1, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-03-26T18:18:22.536 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 115, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 25, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 12, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 12, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 470, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 0, 2024-03-26T18:18:22.537 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 129 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 1, 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 17, 2024-03-26T18:18:22.538 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 5, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 494, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 240, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 11992, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 4581446, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 11992, 2024-03-26T18:18:22.539 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 11992, 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 4581446, 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:22.540 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 2983, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 40808244, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2983, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 40808244, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.541 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 2983, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 2983, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 2983, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 2983, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.542 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 240, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 27280, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 240, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:22.543 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 103, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 27280, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x559b6cad4500": { 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 5816, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 40275823, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:22.544 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 5816, 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 1816, 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 40275823, 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:22.545 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-03-26T18:18:23.220 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-03-26T18:18:23.220 INFO:tasks.check_counter:Daemon mds.i mds.imported=27 2024-03-26T18:18:23.220 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-03-26T18:18:23.220 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-26T18:18:23.220 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T18:18:23.220 DEBUG:tasks.check_counter:Getting stats from l 2024-03-26T18:18:23.221 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-26T18:18:23.477 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:18:23.893 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:23 smithi171 ceph-mon[36033]: pgmap v511: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-03-26T18:18:24.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:23 smithi115 ceph-mon[36174]: pgmap v511: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-03-26T18:18:24.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:23 smithi049 ceph-mon[32331]: pgmap v511: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-03-26T18:18:25.421 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-03-26T18:18:25.421 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:18:25.421 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 417, 2024-03-26T18:18:25.421 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 403, 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1722578, 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 258085, 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 16, 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.061385900, 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.016929394, 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.048394691, 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.007358374, 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 397, 2024-03-26T18:18:25.422 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.022197313, 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000055912 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 20, 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000002935, 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000146 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1718064, 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 202624 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.423 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 423, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 837, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 152884, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 638966, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 10, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.060636870, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.023649521, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.029015359, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.004997074, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 840, 2024-03-26T18:18:25.424 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.079311804, 2024-03-26T18:18:25.425 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000094418 2024-03-26T18:18:25.425 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.425 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:25.425 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1158, 2024-03-26T18:18:25.425 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000293637, 2024-03-26T18:18:25.425 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000253 2024-03-26T18:18:25.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.426 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 6960, 2024-03-26T18:18:25.426 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 147856 2024-03-26T18:18:25.426 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.426 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:18:25.426 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 24, 2024-03-26T18:18:25.426 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 222, 2024-03-26T18:18:25.426 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 6215, 2024-03-26T18:18:25.426 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 536932, 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 10, 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.012105158, 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.006718570, 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.003448766, 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000451210, 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 220, 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.043750053, 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000198863 2024-03-26T18:18:25.427 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 526432 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.428 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 54, 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.001653435, 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000030619 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2, 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000005442, 2024-03-26T18:18:25.429 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000002721 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 8, 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 5, 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5, 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.002019550, 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000403910 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 3, 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-03-26T18:18:25.430 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 11, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 14, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 0, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 14, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 14, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 3, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 3, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 2, 2024-03-26T18:18:25.431 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 10, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 7, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 1, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-03-26T18:18:25.432 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 4, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 234, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:18:25.433 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 1, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 2, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 1, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 1 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-03-26T18:18:25.434 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 2, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:18:25.435 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 8, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-03-26T18:18:25.436 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 8, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 2, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 2, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 2, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 4196655, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 0, 2024-03-26T18:18:25.437 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 7, 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.013060987, 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001865855 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 0 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 17, 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 17, 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 0, 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 15, 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 16, 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 1, 2024-03-26T18:18:25.438 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 14, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 14, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 0, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 3, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 3, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 50444, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 223612 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 9, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 8, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 33, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-03-26T18:18:25.439 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 4, 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.001839522, 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000459880 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.440 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.441 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1, 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000180028, 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000180028 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-03-26T18:18:25.442 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.443 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-03-26T18:18:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.444 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.445 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-03-26T18:18:25.446 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.447 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.448 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-03-26T18:18:25.449 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.450 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:18:25.451 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 4, 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 2, 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 2, 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-03-26T18:18:25.452 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 4, 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 1, 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 949, 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:18:25.453 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-03-26T18:18:25.454 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-03-26T18:18:25.455 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 264375, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 92, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 88, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 1, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:18:25.456 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 9424, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 181, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 68509, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 293, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-03-26T18:18:25.457 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 46, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 13208, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 33, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 46, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 46, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.210995781, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.004586864 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-03-26T18:18:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 46, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 70 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 46, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 0, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 46, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 0, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 10, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 0, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 7, 2024-03-26T18:18:25.459 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 8, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 6, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 20, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:18:25.460 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-03-26T18:18:25.461 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 115, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 25, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 9, 2024-03-26T18:18:25.462 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 14, 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 5, 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 18, 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 0, 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 1 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 1, 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-03-26T18:18:25.463 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-03-26T18:18:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-03-26T18:18:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-03-26T18:18:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:18:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-03-26T18:18:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-03-26T18:18:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-03-26T18:18:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-03-26T18:18:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-03-26T18:18:25.464 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 860, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1780349, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 860, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 860, 2024-03-26T18:18:25.465 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1780349, 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:25.466 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 46, 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 13208, 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 46, 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 13208, 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:25.467 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 46, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 46, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 46, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 46, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.468 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-03-26T18:18:25.469 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x555a1e204500": { 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 8, 2024-03-26T18:18:25.470 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 2351, 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 8, 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 7, 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 2351, 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-03-26T18:18:25.471 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-03-26T18:18:26.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:25 smithi171 ceph-mon[36033]: pgmap v512: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-26T18:18:26.193 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-26T18:18:26.193 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-26T18:18:26.193 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-26T18:18:26.194 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-26T18:18:26.194 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-26T18:18:26.194 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-26T18:18:26.205 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-26T18:18:26.206 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T18:18:26.206 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T18:18:26.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:25 smithi115 ceph-mon[36174]: pgmap v512: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-26T18:18:26.244 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:25 smithi049 ceph-mon[32331]: pgmap v512: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-26T18:18:26.248 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-03-26T18:18:26.248 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T18:18:26.309 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:18:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 -- 172.21.15.49:0/2967817506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6c1007c0 msgr2=0x7ffa6c100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 --2- 172.21.15.49:0/2967817506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6c1007c0 0x7ffa6c100c40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009930 tx=0x7ffa6002bc30 comp rx=0 tx=0).stop 2024-03-26T18:18:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 -- 172.21.15.49:0/2967817506 shutdown_connections 2024-03-26T18:18:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 --2- 172.21.15.49:0/2967817506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa6c101180 0x7ffa6c10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 --2- 172.21.15.49:0/2967817506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6c1007c0 0x7ffa6c100c40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 --2- 172.21.15.49:0/2967817506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c1067d0 0x7ffa6c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 -- 172.21.15.49:0/2967817506 >> 172.21.15.49:0/2967817506 conn(0x7ffa6c0778c0 msgr2=0x7ffa6c077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:26.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 -- 172.21.15.49:0/2967817506 shutdown_connections 2024-03-26T18:18:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 -- 172.21.15.49:0/2967817506 wait complete. 2024-03-26T18:18:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 Processor -- start 2024-03-26T18:18:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.506+0000 7ffa7320b640 1 -- start start 2024-03-26T18:18:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa7320b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6c1007c0 0x7ffa6c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa7320b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa6c101180 0x7ffa6c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa7320b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c1067d0 0x7ffa6c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa7320b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c110be0 con 0x7ffa6c1007c0 2024-03-26T18:18:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa7320b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c110a60 con 0x7ffa6c1067d0 2024-03-26T18:18:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa7320b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c110d60 con 0x7ffa6c101180 2024-03-26T18:18:26.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa70fa7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6c1007c0 0x7ffa6c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa70fa7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6c1007c0 0x7ffa6c198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51500/0 (socket says 172.21.15.49:51500) 2024-03-26T18:18:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa70fa7640 1 -- 172.21.15.49:0/2751220193 learned_addr learned my addr 172.21.15.49:0/2751220193 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa6bfff640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa6c101180 0x7ffa6c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa717a8640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c1067d0 0x7ffa6c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa70fa7640 1 -- 172.21.15.49:0/2751220193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa6c101180 msgr2=0x7ffa6c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa70fa7640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa6c101180 0x7ffa6c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa70fa7640 1 -- 172.21.15.49:0/2751220193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c1067d0 msgr2=0x7ffa6c19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa70fa7640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c1067d0 0x7ffa6c19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa70fa7640 1 -- 172.21.15.49:0/2751220193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c19d150 con 0x7ffa6c1007c0 2024-03-26T18:18:26.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa70fa7640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6c1007c0 0x7ffa6c198160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400e9f0 tx=0x7ffa6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa69ffb640 1 -- 172.21.15.49:0/2751220193 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400cd80 con 0x7ffa6c1007c0 2024-03-26T18:18:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa69ffb640 1 -- 172.21.15.49:0/2751220193 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa64010910 con 0x7ffa6c1007c0 2024-03-26T18:18:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa7320b640 1 -- 172.21.15.49:0/2751220193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6c19d440 con 0x7ffa6c1007c0 2024-03-26T18:18:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.507+0000 7ffa69ffb640 1 -- 172.21.15.49:0/2751220193 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64018990 con 0x7ffa6c1007c0 2024-03-26T18:18:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.508+0000 7ffa7320b640 1 -- 172.21.15.49:0/2751220193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6c1a4cd0 con 0x7ffa6c1007c0 2024-03-26T18:18:26.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.508+0000 7ffa7320b640 1 -- 172.21.15.49:0/2751220193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6c0686b0 con 0x7ffa6c1007c0 2024-03-26T18:18:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.510+0000 7ffa69ffb640 1 -- 172.21.15.49:0/2751220193 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7ffa64010ac0 con 0x7ffa6c1007c0 2024-03-26T18:18:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.510+0000 7ffa69ffb640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7ffa540a3090 0x7ffa540a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.511+0000 7ffa6bfff640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7ffa540a3090 0x7ffa540a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.511+0000 7ffa69ffb640 1 -- 172.21.15.49:0/2751220193 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7ffa640d0260 con 0x7ffa6c1007c0 2024-03-26T18:18:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.511+0000 7ffa6bfff640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7ffa540a3090 0x7ffa540a5550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005ab0 tx=0x7ffa60005fd0 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.514+0000 7ffa69ffb640 1 -- 172.21.15.49:0/2751220193 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffa6409c340 con 0x7ffa6c1007c0 2024-03-26T18:18:26.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.699+0000 7ffa7320b640 1 -- 172.21.15.49:0/2751220193 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa6c1994b0 con 0x7ffa6c1007c0 2024-03-26T18:18:26.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.700+0000 7ffa69ffb640 1 -- 172.21.15.49:0/2751220193 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+25237 (secure 0 0 0) 0x7ffa640a03e0 con 0x7ffa6c1007c0 2024-03-26T18:18:26.676 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:18:26.676 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":115,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T18:18:17.721251+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6801/3127469495":"2024-03-27T18:02:05.919612+0000","172.21.15.49:6800/3127469495":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/1873884295":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/1574907427":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1579518314":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/625009684":"2024-03-27T18:02:05.919612+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:18:26.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 -- 172.21.15.49:0/2751220193 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7ffa540a3090 msgr2=0x7ffa540a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:26.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7ffa540a3090 0x7ffa540a5550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005ab0 tx=0x7ffa60005fd0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 -- 172.21.15.49:0/2751220193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6c1007c0 msgr2=0x7ffa6c198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6c1007c0 0x7ffa6c198160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400e9f0 tx=0x7ffa6400eeb0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 -- 172.21.15.49:0/2751220193 shutdown_connections 2024-03-26T18:18:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7ffa540a3090 0x7ffa540a5550 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa6c101180 0x7ffa6c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa6c1067d0 0x7ffa6c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 --2- 172.21.15.49:0/2751220193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa6c1007c0 0x7ffa6c198160 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 -- 172.21.15.49:0/2751220193 >> 172.21.15.49:0/2751220193 conn(0x7ffa6c0778c0 msgr2=0x7ffa6c068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 -- 172.21.15.49:0/2751220193 shutdown_connections 2024-03-26T18:18:26.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.702+0000 7ffa7320b640 1 -- 172.21.15.49:0/2751220193 wait complete. 2024-03-26T18:18:26.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:55:11.166596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:59:45.382542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:59:46.221139+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T17:59:49.890383+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T18:18:26.690 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:18:26.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.826+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/1377820871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b48107910 msgr2=0x7f6b48107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:26.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.826+0000 7f6b4e0a1640 1 --2- 172.21.15.49:0/1377820871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b48107910 0x7f6b48107cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009930 tx=0x7f6b4002bc30 comp rx=0 tx=0).stop 2024-03-26T18:18:26.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.826+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/1377820871 shutdown_connections 2024-03-26T18:18:26.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.826+0000 7f6b4e0a1640 1 --2- 172.21.15.49:0/1377820871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b48069480 0x7f6b4810e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.826+0000 7f6b4e0a1640 1 --2- 172.21.15.49:0/1377820871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b48068ac0 0x7f6b48068f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.826+0000 7f6b4e0a1640 1 --2- 172.21.15.49:0/1377820871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b48107910 0x7f6b48107cf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/1377820871 >> 172.21.15.49:0/1377820871 conn(0x7f6b480778c0 msgr2=0x7f6b48077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:26.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/1377820871 shutdown_connections 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/1377820871 wait complete. 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 Processor -- start 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 -- start start 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b48068ac0 0x7f6b48100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b48069480 0x7f6b48100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b48107910 0x7f6b48101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b48111c20 con 0x7f6b48068ac0 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f6b48111aa0 con 0x7f6b48069480 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.827+0000 7f6b4e0a1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6b48111da0 con 0x7f6b48107910 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b48068ac0 0x7f6b48100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b477fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b48068ac0 0x7f6b48100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51510/0 (socket says 172.21.15.49:51510) 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b477fe640 1 -- 172.21.15.49:0/4063426367 learned_addr learned my addr 172.21.15.49:0/4063426367 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b47fff640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b48107910 0x7f6b48101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:26.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b46ffd640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b48069480 0x7f6b48100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:26.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b477fe640 1 -- 172.21.15.49:0/4063426367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b48107910 msgr2=0x7f6b48101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:26.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b477fe640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b48107910 0x7f6b48101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b477fe640 1 -- 172.21.15.49:0/4063426367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b48069480 msgr2=0x7f6b48100cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:26.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b477fe640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b48069480 0x7f6b48100cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:26.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b477fe640 1 -- 172.21.15.49:0/4063426367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b481a6660 con 0x7f6b48068ac0 2024-03-26T18:18:26.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b477fe640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b48068ac0 0x7f6b48100780 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009230 tx=0x7f6b40004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:26.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b44ff9640 1 -- 172.21.15.49:0/4063426367 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40041070 con 0x7f6b48068ac0 2024-03-26T18:18:26.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/4063426367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b481a68f0 con 0x7f6b48068ac0 2024-03-26T18:18:26.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b44ff9640 1 -- 172.21.15.49:0/4063426367 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b400054e0 con 0x7f6b48068ac0 2024-03-26T18:18:26.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b44ff9640 1 -- 172.21.15.49:0/4063426367 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40003c90 con 0x7f6b48068ac0 2024-03-26T18:18:26.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.828+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/4063426367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b481a6de0 con 0x7f6b48068ac0 2024-03-26T18:18:26.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.829+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/4063426367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b48101cb0 con 0x7f6b48068ac0 2024-03-26T18:18:26.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.831+0000 7f6b44ff9640 1 -- 172.21.15.49:0/4063426367 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f6b40044070 con 0x7f6b48068ac0 2024-03-26T18:18:26.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.831+0000 7f6b44ff9640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f6b300a3090 0x7f6b300a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:26.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.832+0000 7f6b46ffd640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f6b300a3090 0x7f6b300a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:26.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.832+0000 7f6b44ff9640 1 -- 172.21.15.49:0/4063426367 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f6b400f1af0 con 0x7f6b48068ac0 2024-03-26T18:18:26.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.832+0000 7f6b46ffd640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f6b300a3090 0x7f6b300a5550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6b48102210 tx=0x7f6b3c005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:26.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:26.835+0000 7f6b44ff9640 1 -- 172.21.15.49:0/4063426367 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6b400bdc80 con 0x7f6b48068ac0 2024-03-26T18:18:26.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:26 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2751220193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:18:27.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.020+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/4063426367 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b4802d240 con 0x7f6b48068ac0 2024-03-26T18:18:27.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.020+0000 7f6b44ff9640 1 -- 172.21.15.49:0/4063426367 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f6b400c1d20 con 0x7f6b48068ac0 2024-03-26T18:18:27.054 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-03-26T18:18:27.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/4063426367 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f6b300a3090 msgr2=0x7f6b300a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f6b300a3090 0x7f6b300a5550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6b48102210 tx=0x7f6b3c005f90 comp rx=0 tx=0).stop 2024-03-26T18:18:27.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/4063426367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b48068ac0 msgr2=0x7f6b48100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b48068ac0 0x7f6b48100780 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009230 tx=0x7f6b40004270 comp rx=0 tx=0).stop 2024-03-26T18:18:27.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/4063426367 shutdown_connections 2024-03-26T18:18:27.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f6b300a3090 0x7f6b300a5550 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b48107910 0x7f6b48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b48069480 0x7f6b48100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 --2- 172.21.15.49:0/4063426367 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b48068ac0 0x7f6b48100780 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/4063426367 >> 172.21.15.49:0/4063426367 conn(0x7f6b480778c0 msgr2=0x7f6b480fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:27.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/4063426367 shutdown_connections 2024-03-26T18:18:27.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.022+0000 7f6b4e0a1640 1 -- 172.21.15.49:0/4063426367 wait complete. 2024-03-26T18:18:27.056 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:18:27.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:26 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2751220193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:18:27.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 -- 172.21.15.49:0/3437040579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14dc1011a0 msgr2=0x7f14dc10d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 --2- 172.21.15.49:0/3437040579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14dc1011a0 0x7f14dc10d6b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009cd0 tx=0x7f14d402c0f0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 -- 172.21.15.49:0/3437040579 shutdown_connections 2024-03-26T18:18:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 --2- 172.21.15.49:0/3437040579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc1007e0 0x7f14dc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 --2- 172.21.15.49:0/3437040579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14dc1011a0 0x7f14dc10d6b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 --2- 172.21.15.49:0/3437040579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14dc1067f0 0x7f14dc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 -- 172.21.15.49:0/3437040579 >> 172.21.15.49:0/3437040579 conn(0x7f14dc077920 msgr2=0x7f14dc077d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:27.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 -- 172.21.15.49:0/3437040579 shutdown_connections 2024-03-26T18:18:27.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 -- 172.21.15.49:0/3437040579 wait complete. 2024-03-26T18:18:27.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 Processor -- start 2024-03-26T18:18:27.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.194+0000 7f14e33e6640 1 -- start start 2024-03-26T18:18:27.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e33e6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc1007e0 0x7f14dc198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e33e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14dc1011a0 0x7f14dc199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e33e6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14dc1067f0 0x7f14dc1a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e33e6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc110c00 con 0x7f14dc1067f0 2024-03-26T18:18:27.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e33e6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc110a80 con 0x7f14dc1011a0 2024-03-26T18:18:27.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e33e6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc110d80 con 0x7f14dc1007e0 2024-03-26T18:18:27.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1983640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14dc1067f0 0x7f14dc1a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1182640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc1007e0 0x7f14dc198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1182640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc1007e0 0x7f14dc198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:39716/0 (socket says 172.21.15.49:39716) 2024-03-26T18:18:27.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1983640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14dc1067f0 0x7f14dc1a3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51538/0 (socket says 172.21.15.49:51538) 2024-03-26T18:18:27.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1182640 1 -- 172.21.15.49:0/3396219432 learned_addr learned my addr 172.21.15.49:0/3396219432 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:27.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e0981640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14dc1011a0 0x7f14dc199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1983640 1 -- 172.21.15.49:0/3396219432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc1007e0 msgr2=0x7f14dc198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1983640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc1007e0 0x7f14dc198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1983640 1 -- 172.21.15.49:0/3396219432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14dc1011a0 msgr2=0x7f14dc199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1983640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14dc1011a0 0x7f14dc199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1983640 1 -- 172.21.15.49:0/3396219432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc1a4500 con 0x7f14dc1067f0 2024-03-26T18:18:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e1983640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14dc1067f0 0x7f14dc1a3fc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f14d402c630 tx=0x7f14d4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14d27fc640 1 -- 172.21.15.49:0/3396219432 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4041020 con 0x7f14dc1067f0 2024-03-26T18:18:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14d27fc640 1 -- 172.21.15.49:0/3396219432 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d4034cd0 con 0x7f14dc1067f0 2024-03-26T18:18:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14d27fc640 1 -- 172.21.15.49:0/3396219432 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4039040 con 0x7f14dc1067f0 2024-03-26T18:18:27.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.195+0000 7f14e33e6640 1 -- 172.21.15.49:0/3396219432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc1a46d0 con 0x7f14dc1067f0 2024-03-26T18:18:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.196+0000 7f14e33e6640 1 -- 172.21.15.49:0/3396219432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc1a4b40 con 0x7f14dc1067f0 2024-03-26T18:18:27.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.196+0000 7f14e33e6640 1 -- 172.21.15.49:0/3396219432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14dc068b00 con 0x7f14dc1067f0 2024-03-26T18:18:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.198+0000 7f14d27fc640 1 -- 172.21.15.49:0/3396219432 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f14d4044020 con 0x7f14dc1067f0 2024-03-26T18:18:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.198+0000 7f14d27fc640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f14c80a3090 0x7f14c80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.199+0000 7f14e1182640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f14c80a3090 0x7f14c80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.199+0000 7f14d27fc640 1 -- 172.21.15.49:0/3396219432 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f14d40f1840 con 0x7f14dc1067f0 2024-03-26T18:18:27.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.199+0000 7f14e1182640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f14c80a3090 0x7f14c80a5550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009c10 tx=0x7f14d8005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:27.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.202+0000 7f14d27fc640 1 -- 172.21.15.49:0/3396219432 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f14d404b020 con 0x7f14dc1067f0 2024-03-26T18:18:27.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:26 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2751220193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:18:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.386+0000 7f14e33e6640 1 -- 172.21.15.49:0/3396219432 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14dc108d50 con 0x7f14dc1067f0 2024-03-26T18:18:27.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.387+0000 7f14d27fc640 1 -- 172.21.15.49:0/3396219432 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7f14d40be080 con 0x7f14dc1067f0 2024-03-26T18:18:27.362 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:18:27.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 -- 172.21.15.49:0/3396219432 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f14c80a3090 msgr2=0x7f14c80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f14c80a3090 0x7f14c80a5550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009c10 tx=0x7f14d8005f90 comp rx=0 tx=0).stop 2024-03-26T18:18:27.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 -- 172.21.15.49:0/3396219432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14dc1067f0 msgr2=0x7f14dc1a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14dc1067f0 0x7f14dc1a3fc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f14d402c630 tx=0x7f14d4032f20 comp rx=0 tx=0).stop 2024-03-26T18:18:27.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 -- 172.21.15.49:0/3396219432 shutdown_connections 2024-03-26T18:18:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f14c80a3090 0x7f14c80a5550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f14dc1067f0 0x7f14dc1a3fc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14dc1011a0 0x7f14dc199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 --2- 172.21.15.49:0/3396219432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc1007e0 0x7f14dc198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 -- 172.21.15.49:0/3396219432 >> 172.21.15.49:0/3396219432 conn(0x7f14dc077920 msgr2=0x7f14dc103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 -- 172.21.15.49:0/3396219432 shutdown_connections 2024-03-26T18:18:27.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.389+0000 7f14e33e6640 1 -- 172.21.15.49:0/3396219432 wait complete. 2024-03-26T18:18:27.375 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:18:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.512+0000 7f2ecfeea640 1 -- 172.21.15.49:0/3099070067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec806a7b0 msgr2=0x7f2ec810e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.512+0000 7f2ecfeea640 1 --2- 172.21.15.49:0/3099070067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec806a7b0 0x7f2ec810e740 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009cd0 tx=0x7f2eb802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.512+0000 7f2ecfeea640 1 -- 172.21.15.49:0/3099070067 shutdown_connections 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.512+0000 7f2ecfeea640 1 --2- 172.21.15.49:0/3099070067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8069e30 0x7f2ec806a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.512+0000 7f2ecfeea640 1 --2- 172.21.15.49:0/3099070067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec806a7b0 0x7f2ec810e740 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.512+0000 7f2ecfeea640 1 --2- 172.21.15.49:0/3099070067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec8107880 0x7f2ec8107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.512+0000 7f2ecfeea640 1 -- 172.21.15.49:0/3099070067 >> 172.21.15.49:0/3099070067 conn(0x7f2ec80777a0 msgr2=0x7f2ec8077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.512+0000 7f2ecfeea640 1 -- 172.21.15.49:0/3099070067 shutdown_connections 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecfeea640 1 -- 172.21.15.49:0/3099070067 wait complete. 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecfeea640 1 Processor -- start 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecfeea640 1 -- start start 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecfeea640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8069e30 0x7f2ec8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecfeea640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec806a7b0 0x7f2ec81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecfeea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec8107880 0x7f2ec819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecfeea640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8111c40 con 0x7f2ec806a7b0 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecfeea640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8111ac0 con 0x7f2ec8107880 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecfeea640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec8111dc0 con 0x7f2ec8069e30 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecd485640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec806a7b0 0x7f2ec81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecd485640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec806a7b0 0x7f2ec81986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51560/0 (socket says 172.21.15.49:51560) 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecd485640 1 -- 172.21.15.49:0/4229055801 learned_addr learned my addr 172.21.15.49:0/4229055801 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ece487640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec8107880 0x7f2ec819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.513+0000 7f2ecdc86640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8069e30 0x7f2ec8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ecd485640 1 -- 172.21.15.49:0/4229055801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8069e30 msgr2=0x7f2ec8198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ecd485640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8069e30 0x7f2ec8198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ecd485640 1 -- 172.21.15.49:0/4229055801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec8107880 msgr2=0x7f2ec819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ecd485640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec8107880 0x7f2ec819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ecd485640 1 -- 172.21.15.49:0/4229055801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec819d170 con 0x7f2ec806a7b0 2024-03-26T18:18:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ece487640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec8107880 0x7f2ec819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:18:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ecd485640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec806a7b0 0x7f2ec81986c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00e9f0 tx=0x7f2ebc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ec6ffd640 1 -- 172.21.15.49:0/4229055801 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc00cd80 con 0x7f2ec806a7b0 2024-03-26T18:18:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ec6ffd640 1 -- 172.21.15.49:0/4229055801 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ebc005500 con 0x7f2ec806a7b0 2024-03-26T18:18:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ecfeea640 1 -- 172.21.15.49:0/4229055801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec819d460 con 0x7f2ec806a7b0 2024-03-26T18:18:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ec6ffd640 1 -- 172.21.15.49:0/4229055801 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc010690 con 0x7f2ec806a7b0 2024-03-26T18:18:27.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.514+0000 7f2ecfeea640 1 -- 172.21.15.49:0/4229055801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec81a4cf0 con 0x7f2ec806a7b0 2024-03-26T18:18:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.515+0000 7f2ecfeea640 1 -- 172.21.15.49:0/4229055801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec8102fb0 con 0x7f2ec806a7b0 2024-03-26T18:18:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.516+0000 7f2ec6ffd640 1 -- 172.21.15.49:0/4229055801 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f2ebc0046c0 con 0x7f2ec806a7b0 2024-03-26T18:18:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.517+0000 7f2ec6ffd640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f2eb00a3110 0x7f2eb00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.517+0000 7f2ecdc86640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f2eb00a3110 0x7f2eb00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.517+0000 7f2ec6ffd640 1 -- 172.21.15.49:0/4229055801 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f2ebc0d00d0 con 0x7f2ec806a7b0 2024-03-26T18:18:27.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.517+0000 7f2ecdc86640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f2eb00a3110 0x7f2eb00a55d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00095f0 tx=0x7f2ec0005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.520+0000 7f2ec6ffd640 1 -- 172.21.15.49:0/4229055801 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2ebc053740 con 0x7f2ec806a7b0 2024-03-26T18:18:27.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.705+0000 7f2ecfeea640 1 -- 172.21.15.49:0/4229055801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2ec81994d0 con 0x7f2ec806a7b0 2024-03-26T18:18:27.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.706+0000 7f2ec6ffd640 1 -- 172.21.15.49:0/4229055801 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7f2ebc09c800 con 0x7f2ec806a7b0 2024-03-26T18:18:27.681 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-03-26T18:18:27.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 -- 172.21.15.49:0/4229055801 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f2eb00a3110 msgr2=0x7f2eb00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f2eb00a3110 0x7f2eb00a55d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00095f0 tx=0x7f2ec0005f90 comp rx=0 tx=0).stop 2024-03-26T18:18:27.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 -- 172.21.15.49:0/4229055801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec806a7b0 msgr2=0x7f2ec81986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec806a7b0 0x7f2ec81986c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00e9f0 tx=0x7f2ebc00edf0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 -- 172.21.15.49:0/4229055801 shutdown_connections 2024-03-26T18:18:27.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f2eb00a3110 0x7f2eb00a55d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ec8107880 0x7f2ec819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec806a7b0 0x7f2ec81986c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 --2- 172.21.15.49:0/4229055801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8069e30 0x7f2ec8198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 -- 172.21.15.49:0/4229055801 >> 172.21.15.49:0/4229055801 conn(0x7f2ec80777a0 msgr2=0x7f2ec80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:27.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 -- 172.21.15.49:0/4229055801 shutdown_connections 2024-03-26T18:18:27.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.708+0000 7f2ecfeea640 1 -- 172.21.15.49:0/4229055801 wait complete. 2024-03-26T18:18:27.695 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T18:18:27.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.832+0000 7fe96c697640 1 -- 172.21.15.49:0/3183050037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9641011a0 msgr2=0x7fe96410d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.832+0000 7fe96c697640 1 --2- 172.21.15.49:0/3183050037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9641011a0 0x7fe96410d6b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009cd0 tx=0x7fe95c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.832+0000 7fe96c697640 1 -- 172.21.15.49:0/3183050037 shutdown_connections 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.832+0000 7fe96c697640 1 --2- 172.21.15.49:0/3183050037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9641011a0 0x7fe96410d6b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.832+0000 7fe96c697640 1 --2- 172.21.15.49:0/3183050037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9641007e0 0x7fe964100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.832+0000 7fe96c697640 1 --2- 172.21.15.49:0/3183050037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9641067f0 0x7fe964106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.832+0000 7fe96c697640 1 -- 172.21.15.49:0/3183050037 >> 172.21.15.49:0/3183050037 conn(0x7fe964077920 msgr2=0x7fe964077d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.832+0000 7fe96c697640 1 -- 172.21.15.49:0/3183050037 shutdown_connections 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.832+0000 7fe96c697640 1 -- 172.21.15.49:0/3183050037 wait complete. 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96c697640 1 Processor -- start 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96c697640 1 -- start start 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96c697640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9641007e0 0x7fe964198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96c697640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9641011a0 0x7fe9641994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96c697640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9641067f0 0x7fe9641a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96c697640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe964110c00 con 0x7fe9641067f0 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96c697640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fe964110a80 con 0x7fe9641007e0 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96c697640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe964110d80 con 0x7fe9641011a0 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96ac34640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9641067f0 0x7fe9641a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96ac34640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9641067f0 0x7fe9641a4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51586/0 (socket says 172.21.15.49:51586) 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96ac34640 1 -- 172.21.15.49:0/1081218234 learned_addr learned my addr 172.21.15.49:0/1081218234 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96a433640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9641007e0 0x7fe964198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe969c32640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9641011a0 0x7fe9641994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96ac34640 1 -- 172.21.15.49:0/1081218234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9641011a0 msgr2=0x7fe9641994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96ac34640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9641011a0 0x7fe9641994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96ac34640 1 -- 172.21.15.49:0/1081218234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9641007e0 msgr2=0x7fe964198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:27.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.833+0000 7fe96ac34640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9641007e0 0x7fe964198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.834+0000 7fe96ac34640 1 -- 172.21.15.49:0/1081218234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9641a4560 con 0x7fe9641067f0 2024-03-26T18:18:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.834+0000 7fe96ac34640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9641067f0 0x7fe9641a4020 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0348f0 tx=0x7fe95c032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.834+0000 7fe95b7fe640 1 -- 172.21.15.49:0/1081218234 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c041020 con 0x7fe9641067f0 2024-03-26T18:18:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.834+0000 7fe95b7fe640 1 -- 172.21.15.49:0/1081218234 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95c032e80 con 0x7fe9641067f0 2024-03-26T18:18:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.834+0000 7fe96c697640 1 -- 172.21.15.49:0/1081218234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9641a4730 con 0x7fe9641067f0 2024-03-26T18:18:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.834+0000 7fe95b7fe640 1 -- 172.21.15.49:0/1081218234 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c039040 con 0x7fe9641067f0 2024-03-26T18:18:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.834+0000 7fe96c697640 1 -- 172.21.15.49:0/1081218234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641a4bf0 con 0x7fe9641067f0 2024-03-26T18:18:27.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.835+0000 7fe96c697640 1 -- 172.21.15.49:0/1081218234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe964068b00 con 0x7fe9641067f0 2024-03-26T18:18:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.836+0000 7fe95b7fe640 1 -- 172.21.15.49:0/1081218234 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7fe95c044020 con 0x7fe9641067f0 2024-03-26T18:18:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.837+0000 7fe95b7fe640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fe9500a3110 0x7fe9500a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.837+0000 7fe96a433640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fe9500a3110 0x7fe9500a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.837+0000 7fe95b7fe640 1 -- 172.21.15.49:0/1081218234 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7fe95c0f19d0 con 0x7fe9641067f0 2024-03-26T18:18:27.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.837+0000 7fe96a433640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fe9500a3110 0x7fe9500a55d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe960009b50 tx=0x7fe960005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:27.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:27.840+0000 7fe95b7fe640 1 -- 172.21.15.49:0/1081218234 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe95c04b020 con 0x7fe9641067f0 2024-03-26T18:18:27.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:27 smithi049 ceph-mon[32331]: pgmap v513: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-26T18:18:27.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:27 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4063426367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:18:27.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:27 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3396219432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:18:27.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:27 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4229055801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:18:28.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.025+0000 7fe96c697640 1 -- 172.21.15.49:0/1081218234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe964108d50 con 0x7fe9641067f0 2024-03-26T18:18:28.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.025+0000 7fe95b7fe640 1 -- 172.21.15.49:0/1081218234 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v115) ==== 103+0+11 (secure 0 0 0) 0x7fe95c0be210 con 0x7fe9641067f0 2024-03-26T18:18:28.001 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-03-26T18:18:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.027+0000 7fe96c697640 1 -- 172.21.15.49:0/1081218234 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fe9500a3110 msgr2=0x7fe9500a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.027+0000 7fe96c697640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fe9500a3110 0x7fe9500a55d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe960009b50 tx=0x7fe960005f90 comp rx=0 tx=0).stop 2024-03-26T18:18:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.027+0000 7fe96c697640 1 -- 172.21.15.49:0/1081218234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9641067f0 msgr2=0x7fe9641a4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.027+0000 7fe96c697640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9641067f0 0x7fe9641a4020 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0348f0 tx=0x7fe95c032c90 comp rx=0 tx=0).stop 2024-03-26T18:18:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.027+0000 7fe96c697640 1 -- 172.21.15.49:0/1081218234 shutdown_connections 2024-03-26T18:18:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.028+0000 7fe96c697640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fe9500a3110 0x7fe9500a55d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.028+0000 7fe96c697640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9641011a0 0x7fe9641994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.028+0000 7fe96c697640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9641067f0 0x7fe9641a4020 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.028+0000 7fe96c697640 1 --2- 172.21.15.49:0/1081218234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9641007e0 0x7fe964198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.028+0000 7fe96c697640 1 -- 172.21.15.49:0/1081218234 >> 172.21.15.49:0/1081218234 conn(0x7fe964077920 msgr2=0x7fe9641033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:28.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.028+0000 7fe96c697640 1 -- 172.21.15.49:0/1081218234 shutdown_connections 2024-03-26T18:18:28.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.028+0000 7fe96c697640 1 -- 172.21.15.49:0/1081218234 wait complete. 2024-03-26T18:18:28.014 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T18:18:28.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:27 smithi171 ceph-mon[36033]: pgmap v513: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-26T18:18:28.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:27 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4063426367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:18:28.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:27 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3396219432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:18:28.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:27 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4229055801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:18:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 -- 172.21.15.49:0/2168964878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92cc106780 msgr2=0x7f92cc106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 --2- 172.21.15.49:0/2168964878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92cc106780 0x7f92cc106b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009930 tx=0x7f92bc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:18:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 -- 172.21.15.49:0/2168964878 shutdown_connections 2024-03-26T18:18:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 --2- 172.21.15.49:0/2168964878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92cc100770 0x7f92cc100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 --2- 172.21.15.49:0/2168964878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92cc101130 0x7f92cc10d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 --2- 172.21.15.49:0/2168964878 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92cc106780 0x7f92cc106b60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 -- 172.21.15.49:0/2168964878 >> 172.21.15.49:0/2168964878 conn(0x7f92cc076b40 msgr2=0x7f92cc076f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:28.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 -- 172.21.15.49:0/2168964878 shutdown_connections 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 -- 172.21.15.49:0/2168964878 wait complete. 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 Processor -- start 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.149+0000 7f92d1ebf640 1 -- start start 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92d1ebf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92cc100770 0x7f92cc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92d1ebf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92cc101130 0x7f92cc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92d1ebf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92cc106780 0x7f92cc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92d1ebf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc110b50 con 0x7f92cc101130 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92d1ebf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc1109d0 con 0x7f92cc106780 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92d1ebf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc110cd0 con 0x7f92cc100770 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92cc101130 0x7f92cc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92cb7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92cc100770 0x7f92cc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92cc101130 0x7f92cc1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51608/0 (socket says 172.21.15.49:51608) 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c2ffd640 1 -- 172.21.15.49:0/963102710 learned_addr learned my addr 172.21.15.49:0/963102710 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:28.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92cbfff640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92cc106780 0x7f92cc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c2ffd640 1 -- 172.21.15.49:0/963102710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92cc100770 msgr2=0x7f92cc1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c2ffd640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92cc100770 0x7f92cc1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c2ffd640 1 -- 172.21.15.49:0/963102710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92cc106780 msgr2=0x7f92cc19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c2ffd640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92cc106780 0x7f92cc19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c2ffd640 1 -- 172.21.15.49:0/963102710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc19d190 con 0x7f92cc101130 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c2ffd640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92cc101130 0x7f92cc1986e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f92b400d950 tx=0x7f92b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c97fa640 1 -- 172.21.15.49:0/963102710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b4017070 con 0x7f92cc101130 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c97fa640 1 -- 172.21.15.49:0/963102710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92b4009c30 con 0x7f92cc101130 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92d1ebf640 1 -- 172.21.15.49:0/963102710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92cc19d480 con 0x7f92cc101130 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.150+0000 7f92c97fa640 1 -- 172.21.15.49:0/963102710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b400f6b0 con 0x7f92cc101130 2024-03-26T18:18:28.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.151+0000 7f92d1ebf640 1 -- 172.21.15.49:0/963102710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92cc1a4d10 con 0x7f92cc101130 2024-03-26T18:18:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.151+0000 7f92d1ebf640 1 -- 172.21.15.49:0/963102710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92cc101eb0 con 0x7f92cc101130 2024-03-26T18:18:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.153+0000 7f92c97fa640 1 -- 172.21.15.49:0/963102710 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f92b40109d0 con 0x7f92cc101130 2024-03-26T18:18:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.153+0000 7f92c97fa640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f92b80a3090 0x7f92b80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:28.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.153+0000 7f92cb7fe640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f92b80a3090 0x7f92b80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:28.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.153+0000 7f92c97fa640 1 -- 172.21.15.49:0/963102710 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f92b40cf6f0 con 0x7f92cc101130 2024-03-26T18:18:28.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.154+0000 7f92cb7fe640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f92b80a3090 0x7f92b80a5550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f92bc005ab0 tx=0x7f92bc005fd0 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:28.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.157+0000 7f92c97fa640 1 -- 172.21.15.49:0/963102710 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f92b4009de0 con 0x7f92cc101130 2024-03-26T18:18:28.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:27 smithi115 ceph-mon[36174]: pgmap v513: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-26T18:18:28.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:27 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4063426367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:18:28.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:27 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3396219432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:18:28.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:27 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4229055801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:18:28.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.352+0000 7f92d1ebf640 1 -- 172.21.15.49:0/963102710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f92cc068b00 con 0x7f92cc101130 2024-03-26T18:18:28.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.352+0000 7f92c97fa640 1 -- 172.21.15.49:0/963102710 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v115) ==== 81+0+939 (secure 0 0 0) 0x7f92cc068b00 con 0x7f92cc101130 2024-03-26T18:18:28.327 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/3127469495 2024-03-27T18:02:05.919612+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/3127469495 2024-03-27T18:02:05.919612+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1873884295 2024-03-27T18:02:05.919612+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1574907427 2024-03-27T18:02:05.919612+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/217046874 2024-03-27T17:52:42.621576+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/442612048 2024-03-27T17:52:42.621576+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1502160114 2024-03-27T17:52:42.621576+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/4249557516 2024-03-27T17:52:42.621576+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1579518314 2024-03-27T18:02:05.919612+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2824939577 2024-03-27T17:52:15.448636+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/48778628 2024-03-27T17:52:15.448636+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1915951880 2024-03-27T17:52:15.448636+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/48778628 2024-03-27T17:52:15.448636+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1560035061 2024-03-27T17:52:15.448636+0000 2024-03-26T18:18:28.328 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/625009684 2024-03-27T18:02:05.919612+0000 2024-03-26T18:18:28.329 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/4249557516 2024-03-27T17:52:42.621576+0000 2024-03-26T18:18:28.329 INFO:teuthology.orchestra.run.smithi049.stderr:listed 16 entries 2024-03-26T18:18:28.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 -- 172.21.15.49:0/963102710 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f92b80a3090 msgr2=0x7f92b80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f92b80a3090 0x7f92b80a5550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f92bc005ab0 tx=0x7f92bc005fd0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 -- 172.21.15.49:0/963102710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92cc101130 msgr2=0x7f92cc1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92cc101130 0x7f92cc1986e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f92b400d950 tx=0x7f92b400de10 comp rx=0 tx=0).stop 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 -- 172.21.15.49:0/963102710 shutdown_connections 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f92b80a3090 0x7f92b80a5550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92cc106780 0x7f92cc19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f92cc101130 0x7f92cc1986e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 --2- 172.21.15.49:0/963102710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92cc100770 0x7f92cc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 -- 172.21.15.49:0/963102710 >> 172.21.15.49:0/963102710 conn(0x7f92cc076b40 msgr2=0x7f92cc06a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 -- 172.21.15.49:0/963102710 shutdown_connections 2024-03-26T18:18:28.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.354+0000 7f92d1ebf640 1 -- 172.21.15.49:0/963102710 wait complete. 2024-03-26T18:18:28.341 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-26T18:18:28.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:18:28.341 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /tmp/cephfs 2024-03-26T18:18:28.520 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T18:18:28.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:18:28.521 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-03-26T18:18:28.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:18:28.553 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-03-26T18:18:28.647 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-03-26T18:18:28.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:18:28.647 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-03-26T18:18:28.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:18:28.679 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-03-26T18:18:28.773 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-26T18:18:28.784 INFO:tasks.cephadm:Teardown begin 2024-03-26T18:18:28.784 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-26T18:18:28.784 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:18:28.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.920+0000 7fc72bd37640 1 -- 172.21.15.49:0/3857561056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc72410bf00 msgr2=0x7fc72410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.920+0000 7fc72bd37640 1 --2- 172.21.15.49:0/3857561056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc72410bf00 0x7fc72410e3a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc71402bc30 comp rx=0 tx=0).stop 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.920+0000 7fc72bd37640 1 -- 172.21.15.49:0/3857561056 shutdown_connections 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.920+0000 7fc72bd37640 1 --2- 172.21.15.49:0/3857561056 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc72410bf00 0x7fc72410e3a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.920+0000 7fc72bd37640 1 --2- 172.21.15.49:0/3857561056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc724103450 0x7fc72410b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.920+0000 7fc72bd37640 1 --2- 172.21.15.49:0/3857561056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724102b30 0x7fc724102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.920+0000 7fc72bd37640 1 -- 172.21.15.49:0/3857561056 >> 172.21.15.49:0/3857561056 conn(0x7fc7240778c0 msgr2=0x7fc724077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc72bd37640 1 -- 172.21.15.49:0/3857561056 shutdown_connections 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc72bd37640 1 -- 172.21.15.49:0/3857561056 wait complete. 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc72bd37640 1 Processor -- start 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc72bd37640 1 -- start start 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc72bd37640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724102b30 0x7fc724198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:28.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc72bd37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc724103450 0x7fc724198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc72bd37640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc72410bf00 0x7fc72419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc72bd37640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc724111d00 con 0x7fc724103450 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc72bd37640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fc724111b80 con 0x7fc72410bf00 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc72bd37640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc724111e80 con 0x7fc724102b30 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.921+0000 7fc7292d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc724103450 0x7fc724198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc729ad3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724102b30 0x7fc724198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc729ad3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724102b30 0x7fc724198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.49:33762/0 (socket says 172.21.15.49:33762) 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc7292d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc724103450 0x7fc724198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52010/0 (socket says 172.21.15.49:52010) 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc729ad3640 1 -- 172.21.15.49:0/631862855 learned_addr learned my addr 172.21.15.49:0/631862855 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc72a2d4640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc72410bf00 0x7fc72419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc729ad3640 1 -- 172.21.15.49:0/631862855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc72410bf00 msgr2=0x7fc72419c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc729ad3640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc72410bf00 0x7fc72419c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc729ad3640 1 -- 172.21.15.49:0/631862855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc724103450 msgr2=0x7fc724198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:28.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc729ad3640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc724103450 0x7fc724198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc729ad3640 1 -- 172.21.15.49:0/631862855 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72419d110 con 0x7fc724102b30 2024-03-26T18:18:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc7292d2640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc724103450 0x7fc724198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:18:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc729ad3640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724102b30 0x7fc724198120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00e9f0 tx=0x7fc71c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc722ffd640 1 -- 172.21.15.49:0/631862855 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00cd80 con 0x7fc724102b30 2024-03-26T18:18:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc722ffd640 1 -- 172.21.15.49:0/631862855 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71c010910 con 0x7fc724102b30 2024-03-26T18:18:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc72bd37640 1 -- 172.21.15.49:0/631862855 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72419d400 con 0x7fc724102b30 2024-03-26T18:18:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc722ffd640 1 -- 172.21.15.49:0/631862855 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c018990 con 0x7fc724102b30 2024-03-26T18:18:28.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.922+0000 7fc72bd37640 1 -- 172.21.15.49:0/631862855 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7241a4c90 con 0x7fc724102b30 2024-03-26T18:18:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.923+0000 7fc72bd37640 1 -- 172.21.15.49:0/631862855 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7240686b0 con 0x7fc724102b30 2024-03-26T18:18:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.924+0000 7fc722ffd640 1 -- 172.21.15.49:0/631862855 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7fc71c010ac0 con 0x7fc724102b30 2024-03-26T18:18:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.925+0000 7fc722ffd640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fc70c0a3090 0x7fc70c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.925+0000 7fc7292d2640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fc70c0a3090 0x7fc70c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.925+0000 7fc722ffd640 1 -- 172.21.15.49:0/631862855 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7fc71c0cfec0 con 0x7fc724102b30 2024-03-26T18:18:28.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.925+0000 7fc7292d2640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fc70c0a3090 0x7fc70c0a5550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7180099a0 tx=0x7fc718009500 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:28.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:28.930+0000 7fc722ffd640 1 -- 172.21.15.49:0/631862855 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc71c09bfa0 con 0x7fc724102b30 2024-03-26T18:18:28.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:28 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1081218234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:18:28.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:28 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/963102710' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:18:29.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:28 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1081218234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:18:29.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:28 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/963102710' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:18:29.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.154+0000 7fc72bd37640 1 -- 172.21.15.49:0/631862855 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7241a4e60 con 0x7fc724102b30 2024-03-26T18:18:29.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.156+0000 7fc722ffd640 1 -- 172.21.15.49:0/631862855 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11943 (secure 0 0 0) 0x7fc71c0bc0e0 con 0x7fc724102b30 2024-03-26T18:18:29.130 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:18:29.131 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:59:46.796381+0000","modified":"2024-03-26T18:05:08.307610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24503,"mds_2":24478,"mds_3":24428,"mds_4":24388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24478,24503,24388,24428,24533]},"id":1}]} 2024-03-26T18:18:29.131 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 36 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.157+0000 7fc72bd37640 1 -- 172.21.15.49:0/631862855 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fc70c0a3090 msgr2=0x7fc70c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.157+0000 7fc72bd37640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fc70c0a3090 0x7fc70c0a5550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7180099a0 tx=0x7fc718009500 comp rx=0 tx=0).stop 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.157+0000 7fc72bd37640 1 -- 172.21.15.49:0/631862855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724102b30 msgr2=0x7fc724198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.157+0000 7fc72bd37640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724102b30 0x7fc724198120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00e9f0 tx=0x7fc71c00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.157+0000 7fc72bd37640 1 -- 172.21.15.49:0/631862855 shutdown_connections 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.157+0000 7fc72bd37640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fc70c0a3090 0x7fc70c0a5550 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.157+0000 7fc72bd37640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc72410bf00 0x7fc72419c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.157+0000 7fc72bd37640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc724103450 0x7fc724198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.157+0000 7fc72bd37640 1 --2- 172.21.15.49:0/631862855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724102b30 0x7fc724198120 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.157+0000 7fc72bd37640 1 -- 172.21.15.49:0/631862855 >> 172.21.15.49:0/631862855 conn(0x7fc7240778c0 msgr2=0x7fc7240fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.158+0000 7fc72bd37640 1 -- 172.21.15.49:0/631862855 shutdown_connections 2024-03-26T18:18:29.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.158+0000 7fc72bd37640 1 -- 172.21.15.49:0/631862855 wait complete. 2024-03-26T18:18:29.144 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:18:29.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:28 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1081218234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T18:18:29.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:28 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/963102710' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:18:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 -- 172.21.15.49:0/4124650131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea24107890 msgr2=0x7fea24107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 --2- 172.21.15.49:0/4124650131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea24107890 0x7fea24107c70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fea14009930 tx=0x7fea1402bc30 comp rx=0 tx=0).stop 2024-03-26T18:18:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 -- 172.21.15.49:0/4124650131 shutdown_connections 2024-03-26T18:18:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 --2- 172.21.15.49:0/4124650131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea24101880 0x7fea24101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 --2- 172.21.15.49:0/4124650131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea24102240 0x7fea2410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 --2- 172.21.15.49:0/4124650131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea24107890 0x7fea24107c70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 -- 172.21.15.49:0/4124650131 >> 172.21.15.49:0/4124650131 conn(0x7fea240fd330 msgr2=0x7fea240ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 -- 172.21.15.49:0/4124650131 shutdown_connections 2024-03-26T18:18:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 -- 172.21.15.49:0/4124650131 wait complete. 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 Processor -- start 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.279+0000 7fea2a89a640 1 -- start start 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea2a89a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea24101880 0x7fea24073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea2a89a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea24102240 0x7fea24074100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea2a89a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea24107890 0x7fea24074640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea2a89a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fea24111cb0 con 0x7fea24107890 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea2a89a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fea24111b30 con 0x7fea24102240 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea2a89a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea24111e30 con 0x7fea24101880 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea28e37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea24107890 0x7fea24074640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea28e37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea24107890 0x7fea24074640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52020/0 (socket says 172.21.15.49:52020) 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea28e37640 1 -- 172.21.15.49:0/4265568612 learned_addr learned my addr 172.21.15.49:0/4265568612 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea23fff640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea24101880 0x7fea24073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea237fe640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea24102240 0x7fea24074100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea28e37640 1 -- 172.21.15.49:0/4265568612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea24101880 msgr2=0x7fea24073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea28e37640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea24101880 0x7fea24073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea28e37640 1 -- 172.21.15.49:0/4265568612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea24102240 msgr2=0x7fea24074100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea28e37640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea24102240 0x7fea24074100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea28e37640 1 -- 172.21.15.49:0/4265568612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea241a66b0 con 0x7fea24107890 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.280+0000 7fea28e37640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea24107890 0x7fea24074640 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00eb10 tx=0x7fea1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.281+0000 7fea217fa640 1 -- 172.21.15.49:0/4265568612 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c004b20 con 0x7fea24107890 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.281+0000 7fea2a89a640 1 -- 172.21.15.49:0/4265568612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea241a69a0 con 0x7fea24107890 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.281+0000 7fea217fa640 1 -- 172.21.15.49:0/4265568612 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c004cc0 con 0x7fea24107890 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.281+0000 7fea217fa640 1 -- 172.21.15.49:0/4265568612 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c010760 con 0x7fea24107890 2024-03-26T18:18:29.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.281+0000 7fea2a89a640 1 -- 172.21.15.49:0/4265568612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea241a6ea0 con 0x7fea24107890 2024-03-26T18:18:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.282+0000 7fea2a89a640 1 -- 172.21.15.49:0/4265568612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea24068b00 con 0x7fea24107890 2024-03-26T18:18:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.283+0000 7fea217fa640 1 -- 172.21.15.49:0/4265568612 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7fea1c005020 con 0x7fea24107890 2024-03-26T18:18:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.284+0000 7fea217fa640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fea0c0a3090 0x7fea0c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.284+0000 7fea23fff640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fea0c0a3090 0x7fea0c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.284+0000 7fea217fa640 1 -- 172.21.15.49:0/4265568612 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7fea1c0d00c0 con 0x7fea24107890 2024-03-26T18:18:29.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.284+0000 7fea23fff640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fea0c0a3090 0x7fea0c0a5550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea14009900 tx=0x7fea1402df90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:29.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.288+0000 7fea217fa640 1 -- 172.21.15.49:0/4265568612 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fea1c09c7f0 con 0x7fea24107890 2024-03-26T18:18:29.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.509+0000 7fea2a89a640 1 -- 172.21.15.49:0/4265568612 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea24109df0 con 0x7fea24107890 2024-03-26T18:18:29.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.510+0000 7fea217fa640 1 -- 172.21.15.49:0/4265568612 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11943 (secure 0 0 0) 0x7fea1c09c0f0 con 0x7fea24107890 2024-03-26T18:18:29.485 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:18:29.485 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3996463514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3996463514},{"type":"v1","addr":"172.21.15.49:6835","nonce":3996463514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2438318193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2438318193},{"type":"v1","addr":"172.21.15.49:6837","nonce":2438318193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:59:46.796381+0000","modified":"2024-03-26T18:05:08.307610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24503,"mds_2":24478,"mds_3":24428,"mds_4":24388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/906046850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":906046850},{"type":"v1","addr":"172.21.15.49:6839","nonce":906046850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/935071179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":935071179},{"type":"v1","addr":"172.21.15.49:6841","nonce":935071179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24388":{"gid":24388,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6833/2067087026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2067087026},{"type":"v1","addr":"172.21.15.171:6833","nonce":2067087026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.115:6833/3604276420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3604276420},{"type":"v1","addr":"172.21.15.115:6833","nonce":3604276420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/3854985421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":3854985421},{"type":"v1","addr":"172.21.15.171:6835","nonce":3854985421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2428288879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2428288879},{"type":"v1","addr":"172.21.15.171:6837","nonce":2428288879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/4085026954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4085026954},{"type":"v1","addr":"172.21.15.115:6835","nonce":4085026954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.171:6839/1082475840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1082475840},{"type":"v1","addr":"172.21.15.171:6839","nonce":1082475840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.115:6837/1993258438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1993258438},{"type":"v1","addr":"172.21.15.115:6837","nonce":1993258438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/1435040687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":1435040687},{"type":"v1","addr":"172.21.15.115:6839","nonce":1435040687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24533,"qdb_cluster":[24478,24503,24388,24428,24533]},"id":1}]} 2024-03-26T18:18:29.485 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 36 2024-03-26T18:18:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.511+0000 7fea2a89a640 1 -- 172.21.15.49:0/4265568612 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fea0c0a3090 msgr2=0x7fea0c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.511+0000 7fea2a89a640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fea0c0a3090 0x7fea0c0a5550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea14009900 tx=0x7fea1402df90 comp rx=0 tx=0).stop 2024-03-26T18:18:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.511+0000 7fea2a89a640 1 -- 172.21.15.49:0/4265568612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea24107890 msgr2=0x7fea24074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.511+0000 7fea2a89a640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea24107890 0x7fea24074640 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00eb10 tx=0x7fea1c00c2d0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.512+0000 7fea2a89a640 1 -- 172.21.15.49:0/4265568612 shutdown_connections 2024-03-26T18:18:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.512+0000 7fea2a89a640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fea0c0a3090 0x7fea0c0a5550 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.512+0000 7fea2a89a640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea24107890 0x7fea24074640 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.512+0000 7fea2a89a640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea24102240 0x7fea24074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.512+0000 7fea2a89a640 1 --2- 172.21.15.49:0/4265568612 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea24101880 0x7fea24073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.512+0000 7fea2a89a640 1 -- 172.21.15.49:0/4265568612 >> 172.21.15.49:0/4265568612 conn(0x7fea240fd330 msgr2=0x7fea240fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:29.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.512+0000 7fea2a89a640 1 -- 172.21.15.49:0/4265568612 shutdown_connections 2024-03-26T18:18:29.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.512+0000 7fea2a89a640 1 -- 172.21.15.49:0/4265568612 wait complete. 2024-03-26T18:18:29.498 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:18:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.632+0000 7fa4ab50c640 1 -- 172.21.15.49:0/2334593569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a4101200 msgr2=0x7fa4a410d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.632+0000 7fa4ab50c640 1 --2- 172.21.15.49:0/2334593569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a4101200 0x7fa4a410d6c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009ca0 tx=0x7fa49c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.632+0000 7fa4ab50c640 1 -- 172.21.15.49:0/2334593569 shutdown_connections 2024-03-26T18:18:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.632+0000 7fa4ab50c640 1 --2- 172.21.15.49:0/2334593569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4a4100840 0x7fa4a4100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.632+0000 7fa4ab50c640 1 --2- 172.21.15.49:0/2334593569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a4101200 0x7fa4a410d6c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.632+0000 7fa4ab50c640 1 --2- 172.21.15.49:0/2334593569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4106800 0x7fa4a4106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.632+0000 7fa4ab50c640 1 -- 172.21.15.49:0/2334593569 >> 172.21.15.49:0/2334593569 conn(0x7fa4a4077920 msgr2=0x7fa4a4077d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4ab50c640 1 -- 172.21.15.49:0/2334593569 shutdown_connections 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4ab50c640 1 -- 172.21.15.49:0/2334593569 wait complete. 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4ab50c640 1 Processor -- start 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4ab50c640 1 -- start start 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4ab50c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4100840 0x7fa4a4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4ab50c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a4101200 0x7fa4a41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4ab50c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4a4106800 0x7fa4a419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4ab50c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4111d10 con 0x7fa4a4100840 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4ab50c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4111b90 con 0x7fa4a4101200 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4ab50c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4111e90 con 0x7fa4a4106800 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.633+0000 7fa4a92a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4100840 0x7fa4a4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a92a8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4100840 0x7fa4a4198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52046/0 (socket says 172.21.15.49:52046) 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a92a8640 1 -- 172.21.15.49:0/735823067 learned_addr learned my addr 172.21.15.49:0/735823067 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a8aa7640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a4101200 0x7fa4a41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a9aa9640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4a4106800 0x7fa4a419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a92a8640 1 -- 172.21.15.49:0/735823067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4a4106800 msgr2=0x7fa4a419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a92a8640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4a4106800 0x7fa4a419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a92a8640 1 -- 172.21.15.49:0/735823067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a4101200 msgr2=0x7fa4a41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a92a8640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a4101200 0x7fa4a41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a92a8640 1 -- 172.21.15.49:0/735823067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a419d130 con 0x7fa4a4100840 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a8aa7640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a4101200 0x7fa4a41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4a92a8640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4100840 0x7fa4a4198190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000e9f0 tx=0x7fa4a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa49a7fc640 1 -- 172.21.15.49:0/735823067 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000cd80 con 0x7fa4a4100840 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa49a7fc640 1 -- 172.21.15.49:0/735823067 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a0010910 con 0x7fa4a4100840 2024-03-26T18:18:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4ab50c640 1 -- 172.21.15.49:0/735823067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a419d420 con 0x7fa4a4100840 2024-03-26T18:18:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa49a7fc640 1 -- 172.21.15.49:0/735823067 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0018990 con 0x7fa4a4100840 2024-03-26T18:18:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.634+0000 7fa4ab50c640 1 -- 172.21.15.49:0/735823067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a41a4cb0 con 0x7fa4a4100840 2024-03-26T18:18:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.635+0000 7fa4ab50c640 1 -- 172.21.15.49:0/735823067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a4068b00 con 0x7fa4a4100840 2024-03-26T18:18:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.636+0000 7fa49a7fc640 1 -- 172.21.15.49:0/735823067 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7fa4a0010ac0 con 0x7fa4a4100840 2024-03-26T18:18:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.637+0000 7fa49a7fc640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa4900a3090 0x7fa4900a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.637+0000 7fa4a8aa7640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa4900a3090 0x7fa4900a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.637+0000 7fa49a7fc640 1 -- 172.21.15.49:0/735823067 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7fa4a00d0260 con 0x7fa4a4100840 2024-03-26T18:18:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.637+0000 7fa4a8aa7640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa4900a3090 0x7fa4900a5550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4104e30 tx=0x7fa494005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:29.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.641+0000 7fa49a7fc640 1 -- 172.21.15.49:0/735823067 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa4a009c340 con 0x7fa4a4100840 2024-03-26T18:18:29.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.826+0000 7fa4ab50c640 1 -- 172.21.15.49:0/735823067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4a4108d80 con 0x7fa4a4100840 2024-03-26T18:18:29.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.827+0000 7fa49a7fc640 1 -- 172.21.15.49:0/735823067 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+25237 (secure 0 0 0) 0x7fa4a00a03e0 con 0x7fa4a4100840 2024-03-26T18:18:29.803 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-03-26T18:18:29.803 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":115,"fsid":"5a3632a4-eb99-11ee-95d0-87774f69a715","created":"2024-03-26T17:51:42.367519+0000","modified":"2024-03-26T18:18:17.721251+0000","last_up_change":"2024-03-26T17:58:59.645162+0000","last_in_change":"2024-03-26T17:58:39.679699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:55:11.166596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:59:45.382542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:59:46.221139+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T17:59:49.890383+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"22af3c21-ba61-4714-bf75-a3d2761bed59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6803","nonce":1257601781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6805","nonce":1257601781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6809","nonce":1257601781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1257601781},{"type":"v1","addr":"172.21.15.49:6807","nonce":1257601781}]},"public_addr":"172.21.15.49:6803/1257601781","cluster_addr":"172.21.15.49:6805/1257601781","heartbeat_back_addr":"172.21.15.49:6809/1257601781","heartbeat_front_addr":"172.21.15.49:6807/1257601781","state":["exists","up"]},{"osd":1,"uuid":"68c19f4c-7c26-4834-9bcb-d50b17b767e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6801","nonce":1211178238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6803","nonce":1211178238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6807","nonce":1211178238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1211178238},{"type":"v1","addr":"172.21.15.115:6805","nonce":1211178238}]},"public_addr":"172.21.15.115:6801/1211178238","cluster_addr":"172.21.15.115:6803/1211178238","heartbeat_back_addr":"172.21.15.115:6807/1211178238","heartbeat_front_addr":"172.21.15.115:6805/1211178238","state":["exists","up"]},{"osd":2,"uuid":"01428517-2a3a-43fa-88ce-fb0b460d78eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6801","nonce":2543547141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6803","nonce":2543547141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6807","nonce":2543547141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":2543547141},{"type":"v1","addr":"172.21.15.171:6805","nonce":2543547141}]},"public_addr":"172.21.15.171:6801/2543547141","cluster_addr":"172.21.15.171:6803/2543547141","heartbeat_back_addr":"172.21.15.171:6807/2543547141","heartbeat_front_addr":"172.21.15.171:6805/2543547141","state":["exists","up"]},{"osd":3,"uuid":"c1b332db-6ccf-45ea-ae70-bd6471ce4e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6811","nonce":3710621844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6813","nonce":3710621844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6817","nonce":3710621844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3710621844},{"type":"v1","addr":"172.21.15.49:6815","nonce":3710621844}]},"public_addr":"172.21.15.49:6811/3710621844","cluster_addr":"172.21.15.49:6813/3710621844","heartbeat_back_addr":"172.21.15.49:6817/3710621844","heartbeat_front_addr":"172.21.15.49:6815/3710621844","state":["exists","up"]},{"osd":4,"uuid":"09354324-9abe-4d61-975b-7c82c3f57a2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6809","nonce":1015737283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6811","nonce":1015737283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6815","nonce":1015737283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1015737283},{"type":"v1","addr":"172.21.15.115:6813","nonce":1015737283}]},"public_addr":"172.21.15.115:6809/1015737283","cluster_addr":"172.21.15.115:6811/1015737283","heartbeat_back_addr":"172.21.15.115:6815/1015737283","heartbeat_front_addr":"172.21.15.115:6813/1015737283","state":["exists","up"]},{"osd":5,"uuid":"89af3500-86a4-4dcc-a44d-35ba9af96293","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6809","nonce":2358624537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6811","nonce":2358624537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6815","nonce":2358624537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2358624537},{"type":"v1","addr":"172.21.15.171:6813","nonce":2358624537}]},"public_addr":"172.21.15.171:6809/2358624537","cluster_addr":"172.21.15.171:6811/2358624537","heartbeat_back_addr":"172.21.15.171:6815/2358624537","heartbeat_front_addr":"172.21.15.171:6813/2358624537","state":["exists","up"]},{"osd":6,"uuid":"93c76579-30ec-4c2d-a96a-7f939a9cfd3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6819","nonce":2242936117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6821","nonce":2242936117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6825","nonce":2242936117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":2242936117},{"type":"v1","addr":"172.21.15.49:6823","nonce":2242936117}]},"public_addr":"172.21.15.49:6819/2242936117","cluster_addr":"172.21.15.49:6821/2242936117","heartbeat_back_addr":"172.21.15.49:6825/2242936117","heartbeat_front_addr":"172.21.15.49:6823/2242936117","state":["exists","up"]},{"osd":7,"uuid":"7f64173b-87e4-4f35-a827-287345059695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6817","nonce":32789491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6819","nonce":32789491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6823","nonce":32789491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":32789491},{"type":"v1","addr":"172.21.15.115:6821","nonce":32789491}]},"public_addr":"172.21.15.115:6817/32789491","cluster_addr":"172.21.15.115:6819/32789491","heartbeat_back_addr":"172.21.15.115:6823/32789491","heartbeat_front_addr":"172.21.15.115:6821/32789491","state":["exists","up"]},{"osd":8,"uuid":"a1c2ba0d-b239-4b21-9ecf-ab1a4d948977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6817","nonce":4095981725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6819","nonce":4095981725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6823","nonce":4095981725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4095981725},{"type":"v1","addr":"172.21.15.171:6821","nonce":4095981725}]},"public_addr":"172.21.15.171:6817/4095981725","cluster_addr":"172.21.15.171:6819/4095981725","heartbeat_back_addr":"172.21.15.171:6823/4095981725","heartbeat_front_addr":"172.21.15.171:6821/4095981725","state":["exists","up"]},{"osd":9,"uuid":"e0c8e333-fbec-4186-90aa-a713043370e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6827","nonce":1156582500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6829","nonce":1156582500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6833","nonce":1156582500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1156582500},{"type":"v1","addr":"172.21.15.49:6831","nonce":1156582500}]},"public_addr":"172.21.15.49:6827/1156582500","cluster_addr":"172.21.15.49:6829/1156582500","heartbeat_back_addr":"172.21.15.49:6833/1156582500","heartbeat_front_addr":"172.21.15.49:6831/1156582500","state":["exists","up"]},{"osd":10,"uuid":"d4828303-f823-4e3c-8298-33bba2eda98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6825","nonce":2513437602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6827","nonce":2513437602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6831","nonce":2513437602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2513437602},{"type":"v1","addr":"172.21.15.115:6829","nonce":2513437602}]},"public_addr":"172.21.15.115:6825/2513437602","cluster_addr":"172.21.15.115:6827/2513437602","heartbeat_back_addr":"172.21.15.115:6831/2513437602","heartbeat_front_addr":"172.21.15.115:6829/2513437602","state":["exists","up"]},{"osd":11,"uuid":"8f67590c-f8b2-4ffb-84e5-825d76a17804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6825","nonce":4157105640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6827","nonce":4157105640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6831","nonce":4157105640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":4157105640},{"type":"v1","addr":"172.21.15.171:6829","nonce":4157105640}]},"public_addr":"172.21.15.171:6825/4157105640","cluster_addr":"172.21.15.171:6827/4157105640","heartbeat_back_addr":"172.21.15.171:6831/4157105640","heartbeat_front_addr":"172.21.15.171:6829/4157105640","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:19.536528+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:54:43.556540+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:07.005498+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:33.710199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:55:58.602542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:23.285500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:56:49.574245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:14.635839+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:57:39.993645+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:06.601094+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:32.234710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:58:58.102146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:6801/3127469495":"2024-03-27T18:02:05.919612+0000","172.21.15.49:6800/3127469495":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/1873884295":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/1574907427":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/217046874":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/442612048":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1502160114":"2024-03-27T17:52:42.621576+0000","172.21.15.49:6801/4249557516":"2024-03-27T17:52:42.621576+0000","172.21.15.49:0/1579518314":"2024-03-27T18:02:05.919612+0000","172.21.15.49:0/2824939577":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6800/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1915951880":"2024-03-27T17:52:15.448636+0000","172.21.15.49:6801/48778628":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/1560035061":"2024-03-27T17:52:15.448636+0000","172.21.15.49:0/625009684":"2024-03-27T18:02:05.919612+0000","172.21.15.49:6800/4249557516":"2024-03-27T17:52:42.621576+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 -- 172.21.15.49:0/735823067 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa4900a3090 msgr2=0x7fa4900a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa4900a3090 0x7fa4900a5550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4104e30 tx=0x7fa494005f90 comp rx=0 tx=0).stop 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 -- 172.21.15.49:0/735823067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4100840 msgr2=0x7fa4a4198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4100840 0x7fa4a4198190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000e9f0 tx=0x7fa4a000eeb0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 -- 172.21.15.49:0/735823067 shutdown_connections 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fa4900a3090 0x7fa4900a5550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4a4106800 0x7fa4a419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a4101200 0x7fa4a41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 --2- 172.21.15.49:0/735823067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4a4100840 0x7fa4a4198190 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 -- 172.21.15.49:0/735823067 >> 172.21.15.49:0/735823067 conn(0x7fa4a4077920 msgr2=0x7fa4a4102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 -- 172.21.15.49:0/735823067 shutdown_connections 2024-03-26T18:18:29.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.829+0000 7fa4ab50c640 1 -- 172.21.15.49:0/735823067 wait complete. 2024-03-26T18:18:29.816 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-26T18:18:29.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.951+0000 7f1ffcc19640 1 -- 172.21.15.49:0/243530816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ff810bf00 msgr2=0x7f1ff810e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.951+0000 7f1ffcc19640 1 --2- 172.21.15.49:0/243530816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ff810bf00 0x7f1ff810e370 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009ca0 tx=0x7f1ff002c180 comp rx=0 tx=0).stop 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.952+0000 7f1ffcc19640 1 -- 172.21.15.49:0/243530816 shutdown_connections 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.952+0000 7f1ffcc19640 1 --2- 172.21.15.49:0/243530816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ff810bf00 0x7f1ff810e370 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.952+0000 7f1ffcc19640 1 --2- 172.21.15.49:0/243530816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8103450 0x7f1ff810b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.952+0000 7f1ffcc19640 1 --2- 172.21.15.49:0/243530816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ff8102b30 0x7f1ff8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.952+0000 7f1ffcc19640 1 -- 172.21.15.49:0/243530816 >> 172.21.15.49:0/243530816 conn(0x7f1ff8077920 msgr2=0x7f1ff8077d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.952+0000 7f1ffcc19640 1 -- 172.21.15.49:0/243530816 shutdown_connections 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.952+0000 7f1ffcc19640 1 -- 172.21.15.49:0/243530816 wait complete. 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.952+0000 7f1ffcc19640 1 Processor -- start 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.952+0000 7f1ffcc19640 1 -- start start 2024-03-26T18:18:29.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ffcc19640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8102b30 0x7f1ff8198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ffcc19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ff8103450 0x7f1ff8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ffcc19640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ff810bf00 0x7f1ff819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ffcc19640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff8111cd0 con 0x7f1ff8103450 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ffcc19640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff8111b50 con 0x7f1ff8102b30 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ffcc19640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff8111e50 con 0x7f1ff810bf00 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff5d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ff8103450 0x7f1ff8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff5d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ff8103450 0x7f1ff8198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52068/0 (socket says 172.21.15.49:52068) 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff5d9b640 1 -- 172.21.15.49:0/2431050560 learned_addr learned my addr 172.21.15.49:0/2431050560 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff659c640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8102b30 0x7f1ff8198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff6d9d640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ff810bf00 0x7f1ff819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff5d9b640 1 -- 172.21.15.49:0/2431050560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ff810bf00 msgr2=0x7f1ff819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff5d9b640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ff810bf00 0x7f1ff819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff5d9b640 1 -- 172.21.15.49:0/2431050560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8102b30 msgr2=0x7f1ff8198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff5d9b640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8102b30 0x7f1ff8198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff5d9b640 1 -- 172.21.15.49:0/2431050560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff819d0f0 con 0x7f1ff8103450 2024-03-26T18:18:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ff5d9b640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ff8103450 0x7f1ff8198690 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00e9f0 tx=0x7f1fec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1fe77fe640 1 -- 172.21.15.49:0/2431050560 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00cd80 con 0x7f1ff8103450 2024-03-26T18:18:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1fe77fe640 1 -- 172.21.15.49:0/2431050560 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fec010910 con 0x7f1ff8103450 2024-03-26T18:18:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ffcc19640 1 -- 172.21.15.49:0/2431050560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff819d3e0 con 0x7f1ff8103450 2024-03-26T18:18:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1fe77fe640 1 -- 172.21.15.49:0/2431050560 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec018990 con 0x7f1ff8103450 2024-03-26T18:18:29.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.953+0000 7f1ffcc19640 1 -- 172.21.15.49:0/2431050560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff81a4c70 con 0x7f1ff8103450 2024-03-26T18:18:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.954+0000 7f1ffcc19640 1 -- 172.21.15.49:0/2431050560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff8068b00 con 0x7f1ff8103450 2024-03-26T18:18:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.955+0000 7f1fe77fe640 1 -- 172.21.15.49:0/2431050560 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f1fec010ac0 con 0x7f1ff8103450 2024-03-26T18:18:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.956+0000 7f1fe77fe640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f1fe00a3090 0x7f1fe00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.956+0000 7f1ff659c640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f1fe00a3090 0x7f1fe00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.956+0000 7f1fe77fe640 1 -- 172.21.15.49:0/2431050560 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f1fec0cfe40 con 0x7f1ff8103450 2024-03-26T18:18:29.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.956+0000 7f1ff659c640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f1fe00a3090 0x7f1fe00a5550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009bd0 tx=0x7f1fe8005f90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:29.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:29.960+0000 7f1fe77fe640 1 -- 172.21.15.49:0/2431050560 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1fec09bf20 con 0x7f1ff8103450 2024-03-26T18:18:30.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:29 smithi171 ceph-mon[36033]: pgmap v514: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 96 B/s wr, 9 op/s 2024-03-26T18:18:30.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:29 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/631862855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:18:30.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:29 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/4265568612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:18:30.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:29 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/735823067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:18:30.159 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.184+0000 7f1ffcc19640 1 -- 172.21.15.49:0/2431050560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1ff8107050 con 0x7f1ff8103450 2024-03-26T18:18:30.161 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:29 smithi049 ceph-mon[32331]: pgmap v514: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 96 B/s wr, 9 op/s 2024-03-26T18:18:30.161 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:29 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/631862855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:18:30.161 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:29 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/4265568612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:18:30.161 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:29 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/735823067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:18:30.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.186+0000 7f1fe77fe640 1 -- 172.21.15.49:0/2431050560 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f1fec09ffc0 con 0x7f1ff8103450 2024-03-26T18:18:30.162 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-26T18:18:30.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.190+0000 7f1ffcc19640 1 -- 172.21.15.49:0/2431050560 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f1fe00a3090 msgr2=0x7f1fe00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:30.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.190+0000 7f1ffcc19640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f1fe00a3090 0x7f1fe00a5550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009bd0 tx=0x7f1fe8005f90 comp rx=0 tx=0).stop 2024-03-26T18:18:30.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.190+0000 7f1ffcc19640 1 -- 172.21.15.49:0/2431050560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ff8103450 msgr2=0x7f1ff8198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:30.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.190+0000 7f1ffcc19640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ff8103450 0x7f1ff8198690 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00e9f0 tx=0x7f1fec00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:18:30.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.190+0000 7f1ffcc19640 1 -- 172.21.15.49:0/2431050560 shutdown_connections 2024-03-26T18:18:30.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.190+0000 7f1ffcc19640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f1fe00a3090 0x7f1fe00a5550 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:30.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.190+0000 7f1ffcc19640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ff810bf00 0x7f1ff819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:30.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.190+0000 7f1ffcc19640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ff8103450 0x7f1ff8198690 secure :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00e9f0 tx=0x7f1fec00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:18:30.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.190+0000 7f1ffcc19640 1 --2- 172.21.15.49:0/2431050560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8102b30 0x7f1ff8198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:30.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.190+0000 7f1ffcc19640 1 -- 172.21.15.49:0/2431050560 >> 172.21.15.49:0/2431050560 conn(0x7f1ff8077920 msgr2=0x7f1ff80fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:30.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.191+0000 7f1ffcc19640 1 -- 172.21.15.49:0/2431050560 shutdown_connections 2024-03-26T18:18:30.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.191+0000 7f1ffcc19640 1 -- 172.21.15.49:0/2431050560 wait complete. 2024-03-26T18:18:30.182 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-26T18:18:30.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:29 smithi115 ceph-mon[36174]: pgmap v514: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 96 B/s wr, 9 op/s 2024-03-26T18:18:30.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:29 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/631862855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:18:30.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:29 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/4265568612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:18:30.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:29 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/735823067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:18:30.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.326+0000 7feda615b640 1 -- 172.21.15.49:0/2081476320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feda0100730 msgr2=0x7feda0100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:30.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.326+0000 7feda615b640 1 --2- 172.21.15.49:0/2081476320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feda0100730 0x7feda0100bb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fed94009930 tx=0x7fed9402bc30 comp rx=0 tx=0).stop 2024-03-26T18:18:30.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.326+0000 7feda615b640 1 -- 172.21.15.49:0/2081476320 shutdown_connections 2024-03-26T18:18:30.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.326+0000 7feda615b640 1 --2- 172.21.15.49:0/2081476320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feda01010f0 0x7feda010d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:30.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.326+0000 7feda615b640 1 --2- 172.21.15.49:0/2081476320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feda0100730 0x7feda0100bb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:30.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.326+0000 7feda615b640 1 --2- 172.21.15.49:0/2081476320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda0106740 0x7feda0106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:30.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.326+0000 7feda615b640 1 -- 172.21.15.49:0/2081476320 >> 172.21.15.49:0/2081476320 conn(0x7feda00778c0 msgr2=0x7feda0077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:30.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.326+0000 7feda615b640 1 -- 172.21.15.49:0/2081476320 shutdown_connections 2024-03-26T18:18:30.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.326+0000 7feda615b640 1 -- 172.21.15.49:0/2081476320 wait complete. 2024-03-26T18:18:30.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.327+0000 7feda615b640 1 Processor -- start 2024-03-26T18:18:30.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.327+0000 7feda615b640 1 -- start start 2024-03-26T18:18:30.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7feda615b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda0100730 0x7feda0197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:30.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7feda615b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feda01010f0 0x7feda0197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:30.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7feda615b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feda0106740 0x7feda019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:30.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feda01010f0 0x7feda0197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:30.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9effd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feda01010f0 0x7feda0197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52118/0 (socket says 172.21.15.49:52118) 2024-03-26T18:18:30.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9effd640 1 -- 172.21.15.49:0/3991911970 learned_addr learned my addr 172.21.15.49:0/3991911970 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:30.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feda0110b50 con 0x7feda01010f0 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7feda01109d0 con 0x7feda0100730 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7feda0110cd0 con 0x7feda0106740 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9f7fe640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda0100730 0x7feda0197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9ffff640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feda0106740 0x7feda019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9f7fe640 1 -- 172.21.15.49:0/3991911970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feda0106740 msgr2=0x7feda019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9f7fe640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feda0106740 0x7feda019b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9f7fe640 1 -- 172.21.15.49:0/3991911970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feda01010f0 msgr2=0x7feda0197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9f7fe640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feda01010f0 0x7feda0197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9f7fe640 1 -- 172.21.15.49:0/3991911970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda019c020 con 0x7feda0100730 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.328+0000 7fed9effd640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feda01010f0 0x7feda0197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.329+0000 7fed9f7fe640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda0100730 0x7feda0197030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed9000e9f0 tx=0x7fed9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.329+0000 7fed9cff9640 1 -- 172.21.15.49:0/3991911970 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9000cd80 con 0x7feda0100730 2024-03-26T18:18:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.329+0000 7fed9cff9640 1 -- 172.21.15.49:0/3991911970 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed900035f0 con 0x7feda0100730 2024-03-26T18:18:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.329+0000 7fed9cff9640 1 -- 172.21.15.49:0/3991911970 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90010690 con 0x7feda0100730 2024-03-26T18:18:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.329+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda019c310 con 0x7feda0100730 2024-03-26T18:18:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.329+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda01a3ba0 con 0x7feda0100730 2024-03-26T18:18:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.330+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda0101e70 con 0x7feda0100730 2024-03-26T18:18:30.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.331+0000 7fed9cff9640 1 -- 172.21.15.49:0/3991911970 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7fed90003b30 con 0x7feda0100730 2024-03-26T18:18:30.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.332+0000 7fed9cff9640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fed880a3090 0x7fed880a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:30.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.332+0000 7fed9effd640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fed880a3090 0x7fed880a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:30.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.332+0000 7fed9cff9640 1 -- 172.21.15.49:0/3991911970 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9817+0+0 (secure 0 0 0) 0x7fed900cffe0 con 0x7feda0100730 2024-03-26T18:18:30.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.332+0000 7fed9effd640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fed880a3090 0x7fed880a5550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fed94000c00 tx=0x7fed940058d0 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:30.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.335+0000 7fed9cff9640 1 -- 172.21.15.49:0/3991911970 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fed9009c650 con 0x7feda0100730 2024-03-26T18:18:30.494 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a[32327]: 2024-03-26T18:18:30.185+0000 7f80a54c9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T18:18:30.495 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g[73474]: ignoring --setuser ceph since I am not root 2024-03-26T18:18:30.495 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g[73474]: ignoring --setgroup ceph since I am not root 2024-03-26T18:18:30.495 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g[73474]: starting mds.g at 2024-03-26T18:18:30.495 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a[73837]: ignoring --setuser ceph since I am not root 2024-03-26T18:18:30.495 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a[73837]: ignoring --setgroup ceph since I am not root 2024-03-26T18:18:30.495 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-mds[73841]: -- 172.21.15.49:0/1721602947 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564705219a40 con 0x5647051ec800 2024-03-26T18:18:30.495 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a[73837]: starting mds.a at 2024-03-26T18:18:30.495 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a[73837]: 2024-03-26T18:18:30.293+0000 7f65ee0e0700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6838/0: (98) Address already in use 2024-03-26T18:18:30.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:30.594+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7feda0108cf0 con 0x7feda0100730 2024-03-26T18:18:30.598 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c[66804]: ignoring --setuser ceph since I am not root 2024-03-26T18:18:30.598 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c[66804]: ignoring --setgroup ceph since I am not root 2024-03-26T18:18:30.598 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mds[66808]: -- 172.21.15.171:0/1327382691 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d67db6fa40 con 0x55d67db42800 2024-03-26T18:18:30.598 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c[66804]: starting mds.c at 2024-03-26T18:18:30.599 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i[67107]: ignoring --setuser ceph since I am not root 2024-03-26T18:18:30.599 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i[67107]: ignoring --setgroup ceph since I am not root 2024-03-26T18:18:30.599 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mds[67111]: -- 172.21.15.171:0/12773515 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac04553a40 con 0x55ac04526800 2024-03-26T18:18:30.599 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i[67107]: starting mds.i at 2024-03-26T18:18:30.599 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l[66177]: ignoring --setuser ceph since I am not root 2024-03-26T18:18:30.599 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l[66177]: ignoring --setgroup ceph since I am not root 2024-03-26T18:18:30.600 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mds[66181]: -- 172.21.15.171:0/201909639 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56272c8e5a40 con 0x56272c8b8c00 2024-03-26T18:18:30.600 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l[66177]: starting mds.l at 2024-03-26T18:18:30.600 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f[66498]: ignoring --setuser ceph since I am not root 2024-03-26T18:18:30.600 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f[66498]: ignoring --setgroup ceph since I am not root 2024-03-26T18:18:30.600 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mds[66502]: -- 172.21.15.171:0/3074233782 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5578dc611a40 con 0x5578dc5e4c00 2024-03-26T18:18:30.600 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f[66498]: starting mds.f at 2024-03-26T18:18:30.729 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h[66157]: ignoring --setuser ceph since I am not root 2024-03-26T18:18:30.730 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h[66157]: ignoring --setgroup ceph since I am not root 2024-03-26T18:18:30.730 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mds[66161]: -- 172.21.15.115:0/2284108960 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bea5659a40 con 0x55bea562c800 2024-03-26T18:18:30.730 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h[66157]: starting mds.h at 2024-03-26T18:18:30.730 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k[65527]: ignoring --setuser ceph since I am not root 2024-03-26T18:18:30.730 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k[65527]: ignoring --setgroup ceph since I am not root 2024-03-26T18:18:30.730 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k[65527]: starting mds.k at 2024-03-26T18:18:30.731 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-b[66478]: ignoring --setuser ceph since I am not root 2024-03-26T18:18:30.731 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-b[66478]: ignoring --setgroup ceph since I am not root 2024-03-26T18:18:30.731 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mds[66482]: -- 172.21.15.115:0/1110452681 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c1c4a3ba40 con 0x55c1c4a0f000 2024-03-26T18:18:30.731 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-b[66478]: starting mds.b at 2024-03-26T18:18:30.731 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-e[65848]: ignoring --setuser ceph since I am not root 2024-03-26T18:18:30.731 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-e[65848]: ignoring --setgroup ceph since I am not root 2024-03-26T18:18:30.731 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mds[65852]: -- 172.21.15.115:0/2393323533 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596775e1a40 con 0x5596775b4c00 2024-03-26T18:18:30.731 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-e[65848]: starting mds.e at 2024-03-26T18:18:31.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2431050560' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T18:18:31.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mon[36033]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T18:18:31.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mon[36033]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T18:18:31.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2431050560' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T18:18:31.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mon[36033]: osdmap e116: 12 total, 12 up, 12 in 2024-03-26T18:18:31.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mon[36033]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T18:18:31.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mon[36033]: pgmap v516: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-26T18:18:31.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/3991911970' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:18:31.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:30 smithi171 ceph-mon[36033]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:18:31.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.194+0000 7fed9cff9640 1 -- 172.21.15.49:0/3991911970 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7fed9009bf50 con 0x7feda0100730 2024-03-26T18:18:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fed880a3090 msgr2=0x7fed880a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fed880a3090 0x7fed880a5550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fed94000c00 tx=0x7fed940058d0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda0100730 msgr2=0x7feda0197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda0100730 0x7feda0197030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed9000e9f0 tx=0x7fed9000eeb0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 shutdown_connections 2024-03-26T18:18:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fed880a3090 0x7fed880a5550 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feda0106740 0x7feda019b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feda01010f0 0x7feda0197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 --2- 172.21.15.49:0/3991911970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda0100730 0x7feda0197030 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 >> 172.21.15.49:0/3991911970 conn(0x7feda00778c0 msgr2=0x7feda0102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 shutdown_connections 2024-03-26T18:18:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.197+0000 7feda615b640 1 -- 172.21.15.49:0/3991911970 wait complete. 2024-03-26T18:18:31.184 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2431050560' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T18:18:31.184 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-mon[32331]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T18:18:31.184 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-mon[32331]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T18:18:31.184 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2431050560' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T18:18:31.184 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-mon[32331]: osdmap e116: 12 total, 12 up, 12 in 2024-03-26T18:18:31.184 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-mon[32331]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T18:18:31.184 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-mon[32331]: pgmap v516: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-26T18:18:31.184 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/3991911970' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:18:31.184 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:30 smithi049 ceph-mon[32331]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:18:31.185 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-26T18:18:31.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2431050560' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T18:18:31.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mon[36174]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T18:18:31.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mon[36174]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T18:18:31.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2431050560' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T18:18:31.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mon[36174]: osdmap e116: 12 total, 12 up, 12 in 2024-03-26T18:18:31.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mon[36174]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T18:18:31.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mon[36174]: pgmap v516: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-26T18:18:31.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/3991911970' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:18:31.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:30 smithi115 ceph-mon[36174]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:18:31.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.324+0000 7f536d122640 1 -- 172.21.15.49:0/1376167772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368106740 msgr2=0x7f5368106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:31.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.324+0000 7f536d122640 1 --2- 172.21.15.49:0/1376167772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368106740 0x7f5368106b20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f535c009930 tx=0x7f535c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:18:31.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.325+0000 7f536d122640 1 -- 172.21.15.49:0/1376167772 shutdown_connections 2024-03-26T18:18:31.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.325+0000 7f536d122640 1 --2- 172.21.15.49:0/1376167772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5368100730 0x7f5368100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.325+0000 7f536d122640 1 --2- 172.21.15.49:0/1376167772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53681010f0 0x7f536810d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.325+0000 7f536d122640 1 --2- 172.21.15.49:0/1376167772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368106740 0x7f5368106b20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.325+0000 7f536d122640 1 -- 172.21.15.49:0/1376167772 >> 172.21.15.49:0/1376167772 conn(0x7f53680778c0 msgr2=0x7f5368077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:31.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.325+0000 7f536d122640 1 -- 172.21.15.49:0/1376167772 shutdown_connections 2024-03-26T18:18:31.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.325+0000 7f536d122640 1 -- 172.21.15.49:0/1376167772 wait complete. 2024-03-26T18:18:31.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.325+0000 7f536d122640 1 Processor -- start 2024-03-26T18:18:31.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.325+0000 7f536d122640 1 -- start start 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536d122640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5368100730 0x7f53681981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536d122640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53681010f0 0x7f5368198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536d122640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368106740 0x7f536819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536d122640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5368110ba0 con 0x7f5368106740 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536d122640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f5368110a20 con 0x7f5368100730 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536d122640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5368110d20 con 0x7f53681010f0 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368106740 0x7f536819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368106740 0x7f536819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52144/0 (socket says 172.21.15.49:52144) 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536759e640 1 -- 172.21.15.49:0/2194427266 learned_addr learned my addr 172.21.15.49:0/2194427266 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536659c640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53681010f0 0x7f5368198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:31.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f5366d9d640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5368100730 0x7f53681981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536759e640 1 -- 172.21.15.49:0/2194427266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53681010f0 msgr2=0x7f5368198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536759e640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53681010f0 0x7f5368198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536759e640 1 -- 172.21.15.49:0/2194427266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5368100730 msgr2=0x7f53681981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536759e640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5368100730 0x7f53681981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536759e640 1 -- 172.21.15.49:0/2194427266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536819d1c0 con 0x7f5368106740 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.326+0000 7f536759e640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368106740 0x7f536819ca90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f536000cad0 tx=0x7f536000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.327+0000 7f534ffff640 1 -- 172.21.15.49:0/2194427266 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536000ae20 con 0x7f5368106740 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.327+0000 7f534ffff640 1 -- 172.21.15.49:0/2194427266 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5360005020 con 0x7f5368106740 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.327+0000 7f536d122640 1 -- 172.21.15.49:0/2194427266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536819d4b0 con 0x7f5368106740 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.327+0000 7f534ffff640 1 -- 172.21.15.49:0/2194427266 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536000f6f0 con 0x7f5368106740 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.327+0000 7f536d122640 1 -- 172.21.15.49:0/2194427266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a4d40 con 0x7f5368106740 2024-03-26T18:18:31.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.328+0000 7f536d122640 1 -- 172.21.15.49:0/2194427266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53680686b0 con 0x7f5368106740 2024-03-26T18:18:31.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.329+0000 7f534ffff640 1 -- 172.21.15.49:0/2194427266 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f53600051d0 con 0x7f5368106740 2024-03-26T18:18:31.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.329+0000 7f534ffff640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f53500a3090 0x7f53500a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:31.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.329+0000 7f5366d9d640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f53500a3090 0x7f53500a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:31.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.329+0000 7f534ffff640 1 -- 172.21.15.49:0/2194427266 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9817+0+0 (secure 0 0 0) 0x7f53600cfe30 con 0x7f5368106740 2024-03-26T18:18:31.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.330+0000 7f5366d9d640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f53500a3090 0x7f53500a5550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f535c009900 tx=0x7f535c02df90 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.334+0000 7f534ffff640 1 -- 172.21.15.49:0/2194427266 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5360014030 con 0x7f5368106740 2024-03-26T18:18:31.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:31.523+0000 7f536d122640 1 -- 172.21.15.49:0/2194427266 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5368109d30 con 0x7f5368106740 2024-03-26T18:18:32.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: Cluster is now healthy 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6837/1215153795,v1:172.21.15.115:6839/1215153795] up:boot 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6836/3679367902,v1:172.21.15.115:6838/3679367902] up:boot 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6832/2872696870,v1:172.21.15.115:6834/2872696870] up:boot 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.115:6833/4265921004,v1:172.21.15.115:6835/4265921004] up:boot 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.171:6833/2905785924,v1:172.21.15.171:6835/2905785924] up:boot 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.171:6836/2971056862,v1:172.21.15.171:6838/2971056862] up:boot 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.171:6837/2280044143,v1:172.21.15.171:6839/2280044143] up:boot 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.171:6832/2164682115,v1:172.21.15.171:6834/2164682115] up:boot 2024-03-26T18:18:32.099 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.49:6839/2996893902,v1:172.21.15.49:6840/2996893902] up:boot 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: mds.? [v2:172.21.15.49:6838/3220901354,v1:172.21.15.49:6841/3220901354] up:boot 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: fsmap 12 up:standby 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:18:32.100 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:31 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2194427266' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:18:32.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.192+0000 7f534ffff640 1 -- 172.21.15.49:0/2194427266 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) ==== 181+0+0 (secure 0 0 0) 0x7f536009c410 con 0x7f5368106740 2024-03-26T18:18:32.167 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2024-03-26T18:18:32.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.194+0000 7f536d122640 1 -- 172.21.15.49:0/2194427266 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f53500a3090 msgr2=0x7f53500a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:32.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.194+0000 7f536d122640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f53500a3090 0x7f53500a5550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f535c009900 tx=0x7f535c02df90 comp rx=0 tx=0).stop 2024-03-26T18:18:32.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.195+0000 7f536d122640 1 -- 172.21.15.49:0/2194427266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368106740 msgr2=0x7f536819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:32.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.195+0000 7f536d122640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368106740 0x7f536819ca90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f536000cad0 tx=0x7f536000cf90 comp rx=0 tx=0).stop 2024-03-26T18:18:32.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.195+0000 7f536d122640 1 -- 172.21.15.49:0/2194427266 shutdown_connections 2024-03-26T18:18:32.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.195+0000 7f536d122640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f53500a3090 0x7f53500a5550 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:32.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.195+0000 7f536d122640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53681010f0 0x7f5368198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:32.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.195+0000 7f536d122640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5368106740 0x7f536819ca90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:32.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.195+0000 7f536d122640 1 --2- 172.21.15.49:0/2194427266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5368100730 0x7f53681981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:32.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.195+0000 7f536d122640 1 -- 172.21.15.49:0/2194427266 >> 172.21.15.49:0/2194427266 conn(0x7f53680778c0 msgr2=0x7f5368068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:32.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.195+0000 7f536d122640 1 -- 172.21.15.49:0/2194427266 shutdown_connections 2024-03-26T18:18:32.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.195+0000 7f536d122640 1 -- 172.21.15.49:0/2194427266 wait complete. 2024-03-26T18:18:32.184 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: Cluster is now healthy 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6837/1215153795,v1:172.21.15.115:6839/1215153795] up:boot 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6836/3679367902,v1:172.21.15.115:6838/3679367902] up:boot 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6832/2872696870,v1:172.21.15.115:6834/2872696870] up:boot 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.115:6833/4265921004,v1:172.21.15.115:6835/4265921004] up:boot 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.171:6833/2905785924,v1:172.21.15.171:6835/2905785924] up:boot 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.171:6836/2971056862,v1:172.21.15.171:6838/2971056862] up:boot 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.171:6837/2280044143,v1:172.21.15.171:6839/2280044143] up:boot 2024-03-26T18:18:32.185 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.171:6832/2164682115,v1:172.21.15.171:6834/2164682115] up:boot 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.49:6839/2996893902,v1:172.21.15.49:6840/2996893902] up:boot 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: mds.? [v2:172.21.15.49:6838/3220901354,v1:172.21.15.49:6841/3220901354] up:boot 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: fsmap 12 up:standby 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:18:32.186 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:18:32.187 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:31 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2194427266' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:18:32.187 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-26T18:18:32.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T18:18:32.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T18:18:32.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: Cluster is now healthy 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6837/1215153795,v1:172.21.15.115:6839/1215153795] up:boot 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6836/3679367902,v1:172.21.15.115:6838/3679367902] up:boot 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6832/2872696870,v1:172.21.15.115:6834/2872696870] up:boot 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.115:6833/4265921004,v1:172.21.15.115:6835/4265921004] up:boot 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.171:6833/2905785924,v1:172.21.15.171:6835/2905785924] up:boot 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.171:6836/2971056862,v1:172.21.15.171:6838/2971056862] up:boot 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.171:6837/2280044143,v1:172.21.15.171:6839/2280044143] up:boot 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.171:6832/2164682115,v1:172.21.15.171:6834/2164682115] up:boot 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.49:6839/2996893902,v1:172.21.15.49:6840/2996893902] up:boot 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: mds.? [v2:172.21.15.49:6838/3220901354,v1:172.21.15.49:6841/3220901354] up:boot 2024-03-26T18:18:32.230 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: fsmap 12 up:standby 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='mgr.15516 172.21.15.49:0/4248062985' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:18:32.231 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:31 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2194427266' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:18:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.328+0000 7f883f285640 1 -- 172.21.15.49:0/4142483209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8838101130 msgr2=0x7f883810d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.328+0000 7f883f285640 1 --2- 172.21.15.49:0/4142483209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8838101130 0x7f883810d640 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8830009cd0 tx=0x7f883002c0f0 comp rx=0 tx=0).stop 2024-03-26T18:18:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.328+0000 7f883f285640 1 -- 172.21.15.49:0/4142483209 shutdown_connections 2024-03-26T18:18:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.328+0000 7f883f285640 1 --2- 172.21.15.49:0/4142483209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8838101130 0x7f883810d640 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.328+0000 7f883f285640 1 --2- 172.21.15.49:0/4142483209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8838100770 0x7f8838100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.328+0000 7f883f285640 1 --2- 172.21.15.49:0/4142483209 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8838106780 0x7f8838106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.328+0000 7f883f285640 1 -- 172.21.15.49:0/4142483209 >> 172.21.15.49:0/4142483209 conn(0x7f8838076b40 msgr2=0x7f8838076f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.328+0000 7f883f285640 1 -- 172.21.15.49:0/4142483209 shutdown_connections 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.328+0000 7f883f285640 1 -- 172.21.15.49:0/4142483209 wait complete. 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883f285640 1 Processor -- start 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883f285640 1 -- start start 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883f285640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8838100770 0x7f8838198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883f285640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8838101130 0x7f88381985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883f285640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8838106780 0x7f883819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883f285640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8838110b50 con 0x7f8838101130 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883f285640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7f88381109d0 con 0x7f8838106780 2024-03-26T18:18:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883f285640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8838110cd0 con 0x7f8838100770 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883c820640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8838101130 0x7f88381985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883c820640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8838101130 0x7f88381985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52160/0 (socket says 172.21.15.49:52160) 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883c820640 1 -- 172.21.15.49:0/1576843586 learned_addr learned my addr 172.21.15.49:0/1576843586 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883d021640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8838100770 0x7f8838198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883d822640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8838106780 0x7f883819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883c820640 1 -- 172.21.15.49:0/1576843586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8838100770 msgr2=0x7f8838198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883c820640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8838100770 0x7f8838198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883c820640 1 -- 172.21.15.49:0/1576843586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8838106780 msgr2=0x7f883819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883c820640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8838106780 0x7f883819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883c820640 1 -- 172.21.15.49:0/1576843586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883819d070 con 0x7f8838101130 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883d021640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8838100770 0x7f8838198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:18:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.329+0000 7f883d822640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8838106780 0x7f883819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:18:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.330+0000 7f883c820640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8838101130 0x7f88381985c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f882800c890 tx=0x7f882800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.330+0000 7f882e7fc640 1 -- 172.21.15.49:0/1576843586 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828004220 con 0x7f8838101130 2024-03-26T18:18:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.330+0000 7f882e7fc640 1 -- 172.21.15.49:0/1576843586 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88280043c0 con 0x7f8838101130 2024-03-26T18:18:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.330+0000 7f883f285640 1 -- 172.21.15.49:0/1576843586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883819d360 con 0x7f8838101130 2024-03-26T18:18:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.330+0000 7f882e7fc640 1 -- 172.21.15.49:0/1576843586 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88280049e0 con 0x7f8838101130 2024-03-26T18:18:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.330+0000 7f883f285640 1 -- 172.21.15.49:0/1576843586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381a4bf0 con 0x7f8838101130 2024-03-26T18:18:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.331+0000 7f883f285640 1 -- 172.21.15.49:0/1576843586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8838101eb0 con 0x7f8838101130 2024-03-26T18:18:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.332+0000 7f882e7fc640 1 -- 172.21.15.49:0/1576843586 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7f88280079b0 con 0x7f8838101130 2024-03-26T18:18:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.333+0000 7f882e7fc640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f88240a3090 0x7f88240a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.333+0000 7f883d021640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f88240a3090 0x7f88240a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.333+0000 7f882e7fc640 1 -- 172.21.15.49:0/1576843586 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9321+0+0 (secure 0 0 0) 0x7f88280cfcb0 con 0x7f8838101130 2024-03-26T18:18:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.333+0000 7f883d021640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f88240a3090 0x7f88240a5550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8834005e10 tx=0x7f8834005da0 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.336+0000 7f882e7fc640 1 -- 172.21.15.49:0/1576843586 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8828014030 con 0x7f8838101130 2024-03-26T18:18:32.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:32.534+0000 7f883f285640 1 -- 172.21.15.49:0/1576843586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8838068b00 con 0x7f8838101130 2024-03-26T18:18:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.199+0000 7f882e7fc640 1 -- 172.21.15.49:0/1576843586 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) ==== 169+0+0 (secure 0 0 0) 0x7f882809c480 con 0x7f8838101130 2024-03-26T18:18:33.173 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2024-03-26T18:18:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 -- 172.21.15.49:0/1576843586 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f88240a3090 msgr2=0x7f88240a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f88240a3090 0x7f88240a5550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8834005e10 tx=0x7f8834005da0 comp rx=0 tx=0).stop 2024-03-26T18:18:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 -- 172.21.15.49:0/1576843586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8838101130 msgr2=0x7f88381985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8838101130 0x7f88381985c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f882800c890 tx=0x7f882800cd50 comp rx=0 tx=0).stop 2024-03-26T18:18:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 -- 172.21.15.49:0/1576843586 shutdown_connections 2024-03-26T18:18:33.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7f88240a3090 0x7f88240a5550 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8838106780 0x7f883819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8838101130 0x7f88381985c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 --2- 172.21.15.49:0/1576843586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8838100770 0x7f8838198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 -- 172.21.15.49:0/1576843586 >> 172.21.15.49:0/1576843586 conn(0x7f8838076b40 msgr2=0x7f88381044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 -- 172.21.15.49:0/1576843586 shutdown_connections 2024-03-26T18:18:33.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.203+0000 7f883f285640 1 -- 172.21.15.49:0/1576843586 wait complete. 2024-03-26T18:18:33.199 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-26T18:18:33.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 -- 172.21.15.49:0/2268887224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c0101130 msgr2=0x7fd3c010d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:33.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 --2- 172.21.15.49:0/2268887224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c0101130 0x7fd3c010d640 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009cf0 tx=0x7fd3b802e300 comp rx=0 tx=0).stop 2024-03-26T18:18:33.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 -- 172.21.15.49:0/2268887224 shutdown_connections 2024-03-26T18:18:33.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 --2- 172.21.15.49:0/2268887224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c0101130 0x7fd3c010d640 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:33.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 --2- 172.21.15.49:0/2268887224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3c0100770 0x7fd3c0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:33.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 --2- 172.21.15.49:0/2268887224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3c0106780 0x7fd3c0106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:33.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 -- 172.21.15.49:0/2268887224 >> 172.21.15.49:0/2268887224 conn(0x7fd3c0076b40 msgr2=0x7fd3c0076f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:33.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 -- 172.21.15.49:0/2268887224 shutdown_connections 2024-03-26T18:18:33.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 -- 172.21.15.49:0/2268887224 wait complete. 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 Processor -- start 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.346+0000 7fd3c664b640 1 -- start start 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3c664b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c0100770 0x7fd3c0195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3c664b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3c0101130 0x7fd3c01963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3c664b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3c0106780 0x7fd3c019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3c664b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c010fa80 con 0x7fd3c0100770 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3c664b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c010f900 con 0x7fd3c0106780 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3c664b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c010fc00 con 0x7fd3c0101130 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3bffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c0100770 0x7fd3c0195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3bffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c0100770 0x7fd3c0195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52184/0 (socket says 172.21.15.49:52184) 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3bffff640 1 -- 172.21.15.49:0/1871541119 learned_addr learned my addr 172.21.15.49:0/1871541119 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-03-26T18:18:33.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3bf7fe640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3c0101130 0x7fd3c01963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:33.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3c4be8640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3c0106780 0x7fd3c019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:33.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3bffff640 1 -- 172.21.15.49:0/1871541119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3c0101130 msgr2=0x7fd3c01963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:33.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3bffff640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3c0101130 0x7fd3c01963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:33.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3bffff640 1 -- 172.21.15.49:0/1871541119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3c0106780 msgr2=0x7fd3c019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:33.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3bffff640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3c0106780 0x7fd3c019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:33.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3bffff640 1 -- 172.21.15.49:0/1871541119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c019ae90 con 0x7fd3c0100770 2024-03-26T18:18:33.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.347+0000 7fd3bffff640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c0100770 0x7fd3c0195ea0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000d770 tx=0x7fd3b000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:33.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.348+0000 7fd3bd7fa640 1 -- 172.21.15.49:0/1871541119 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0017070 con 0x7fd3c0100770 2024-03-26T18:18:33.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.348+0000 7fd3c664b640 1 -- 172.21.15.49:0/1871541119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c019b180 con 0x7fd3c0100770 2024-03-26T18:18:33.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.348+0000 7fd3bd7fa640 1 -- 172.21.15.49:0/1871541119 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3b0009820 con 0x7fd3c0100770 2024-03-26T18:18:33.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.348+0000 7fd3bd7fa640 1 -- 172.21.15.49:0/1871541119 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0004cd0 con 0x7fd3c0100770 2024-03-26T18:18:33.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.348+0000 7fd3c664b640 1 -- 172.21.15.49:0/1871541119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c01a2a10 con 0x7fd3c0100770 2024-03-26T18:18:33.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.349+0000 7fd3c664b640 1 -- 172.21.15.49:0/1871541119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c0101eb0 con 0x7fd3c0100770 2024-03-26T18:18:33.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.350+0000 7fd3bd7fa640 1 -- 172.21.15.49:0/1871541119 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 26) ==== 137985+0+0 (secure 0 0 0) 0x7fd3b00099d0 con 0x7fd3c0100770 2024-03-26T18:18:33.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.350+0000 7fd3bd7fa640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd3a80a3090 0x7fd3a80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:18:33.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.350+0000 7fd3bf7fe640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd3a80a3090 0x7fd3a80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:18:33.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.350+0000 7fd3bd7fa640 1 -- 172.21.15.49:0/1871541119 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8754+0+0 (secure 0 0 0) 0x7fd3b00cf2d0 con 0x7fd3c0100770 2024-03-26T18:18:33.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.351+0000 7fd3bf7fe640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd3a80a3090 0x7fd3a80a5550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00fd780 tx=0x7fd3b400a2d0 comp rx=0 tx=0).ready entity=mgr.15516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:18:33.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.355+0000 7fd3bd7fa640 1 -- 172.21.15.49:0/1871541119 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd3b009bcd0 con 0x7fd3c0100770 2024-03-26T18:18:33.479 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:33 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/2194427266' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:18:33.479 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:33 smithi115 ceph-mon[36174]: osdmap e117: 12 total, 12 up, 12 in 2024-03-26T18:18:33.479 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:33 smithi115 ceph-mon[36174]: pgmap v518: 97 pgs: 97 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:18:33.479 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:33 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1576843586' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:18:33.495 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:33 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/2194427266' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:18:33.495 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:33 smithi049 ceph-mon[32331]: osdmap e117: 12 total, 12 up, 12 in 2024-03-26T18:18:33.495 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:33 smithi049 ceph-mon[32331]: pgmap v518: 97 pgs: 97 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:18:33.495 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:33 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1576843586' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:18:33.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:33.540+0000 7fd3c664b640 1 -- 172.21.15.49:0/1871541119 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd3c0068b00 con 0x7fd3c0100770 2024-03-26T18:18:33.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:33 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/2194427266' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:18:33.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:33 smithi171 ceph-mon[36033]: osdmap e117: 12 total, 12 up, 12 in 2024-03-26T18:18:33.599 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:33 smithi171 ceph-mon[36033]: pgmap v518: 97 pgs: 97 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:18:33.599 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:33 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1576843586' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:18:34.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.200+0000 7fd3bd7fa640 1 -- 172.21.15.49:0/1871541119 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v119) ==== 178+0+0 (secure 0 0 0) 0x7fd3b009b5d0 con 0x7fd3c0100770 2024-03-26T18:18:34.175 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data_ec' removed 2024-03-26T18:18:34.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.204+0000 7fd3c664b640 1 -- 172.21.15.49:0/1871541119 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd3a80a3090 msgr2=0x7fd3a80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:34.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.204+0000 7fd3c664b640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd3a80a3090 0x7fd3a80a5550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00fd780 tx=0x7fd3b400a2d0 comp rx=0 tx=0).stop 2024-03-26T18:18:34.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.205+0000 7fd3c664b640 1 -- 172.21.15.49:0/1871541119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c0100770 msgr2=0x7fd3c0195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:18:34.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.205+0000 7fd3c664b640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c0100770 0x7fd3c0195ea0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000d770 tx=0x7fd3b000db70 comp rx=0 tx=0).stop 2024-03-26T18:18:34.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.205+0000 7fd3c664b640 1 -- 172.21.15.49:0/1871541119 shutdown_connections 2024-03-26T18:18:34.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.205+0000 7fd3c664b640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.49:6800/2879593470,v1:172.21.15.49:6801/2879593470] conn(0x7fd3a80a3090 0x7fd3a80a5550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:34.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.205+0000 7fd3c664b640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd3c0101130 0x7fd3c01963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:34.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.205+0000 7fd3c664b640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3c0106780 0x7fd3c019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:34.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.205+0000 7fd3c664b640 1 --2- 172.21.15.49:0/1871541119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c0100770 0x7fd3c0195ea0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:18:34.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.205+0000 7fd3c664b640 1 -- 172.21.15.49:0/1871541119 >> 172.21.15.49:0/1871541119 conn(0x7fd3c0076b40 msgr2=0x7fd3c00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:18:34.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.205+0000 7fd3c664b640 1 -- 172.21.15.49:0/1871541119 shutdown_connections 2024-03-26T18:18:34.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-03-26T18:18:34.205+0000 7fd3c664b640 1 -- 172.21.15.49:0/1871541119 wait complete. 2024-03-26T18:18:34.201 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:18:34.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:18:34.301 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:18:34.370 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-26T18:18:34.371 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T18:18:34.404 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T18:18:34.425 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T18:18:34.451 INFO:tasks.cephadm:Stopping all daemons... 2024-03-26T18:18:34.451 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-26T18:18:34.451 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.a 2024-03-26T18:18:34.479 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:34 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1576843586' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:18:34.479 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:34 smithi115 ceph-mon[36174]: osdmap e118: 12 total, 12 up, 12 in 2024-03-26T18:18:34.479 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:34 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1871541119' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:18:34.494 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:34 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1576843586' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:18:34.494 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:34 smithi049 ceph-mon[32331]: osdmap e118: 12 total, 12 up, 12 in 2024-03-26T18:18:34.494 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:34 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1871541119' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:18:34.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:34 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1576843586' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:18:34.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:34 smithi171 ceph-mon[36033]: osdmap e118: 12 total, 12 up, 12 in 2024-03-26T18:18:34.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:34 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1871541119' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:18:34.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:34 smithi049 systemd[1]: Stopping Ceph mon.a for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:18:35.454 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:35 smithi049 ceph-mon[32331]: from='client.? 172.21.15.49:0/1871541119' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:18:35.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:35 smithi049 ceph-mon[32331]: osdmap e119: 12 total, 12 up, 12 in 2024-03-26T18:18:35.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:35 smithi049 ceph-mon[32331]: pgmap v521: 1 pgs: 1 active+clean; 577 KiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:18:35.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:35 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a[32327]: 2024-03-26T18:18:35.224+0000 7f80aacd4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:18:35.455 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:35 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a[32327]: 2024-03-26T18:18:35.224+0000 7f80aacd4700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-26T18:18:35.479 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:35 smithi115 ceph-mon[36174]: from='client.? 172.21.15.49:0/1871541119' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:18:35.479 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:35 smithi115 ceph-mon[36174]: osdmap e119: 12 total, 12 up, 12 in 2024-03-26T18:18:35.479 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:35 smithi115 ceph-mon[36174]: pgmap v521: 1 pgs: 1 active+clean; 577 KiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:18:35.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:35 smithi171 ceph-mon[36033]: from='client.? 172.21.15.49:0/1871541119' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:18:35.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:35 smithi171 ceph-mon[36033]: osdmap e119: 12 total, 12 up, 12 in 2024-03-26T18:18:35.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:35 smithi171 ceph-mon[36033]: pgmap v521: 1 pgs: 1 active+clean; 577 KiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T18:18:35.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:35 smithi049 podman[93012]: 2024-03-26 18:18:35.480983429 +0000 UTC m=+0.414836581 container died f601912b7bc11471815d1a9309e870fb67109684e4edda065dcd5ed7da392b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:18:36.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:35 smithi049 podman[93012]: 2024-03-26 18:18:35.903097313 +0000 UTC m=+0.836950461 container cleanup f601912b7bc11471815d1a9309e870fb67109684e4edda065dcd5ed7da392b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-26T18:18:36.170 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:35 smithi049 bash[93012]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a 2024-03-26T18:18:36.494 INFO:journalctl@ceph.mon.a.smithi049.stdout:Mar 26 18:18:36 smithi049 podman[93023]: 2024-03-26 18:18:36.196670566 +0000 UTC m=+0.709490444 container remove f601912b7bc11471815d1a9309e870fb67109684e4edda065dcd5ed7da392b52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2) 2024-03-26T18:18:36.606 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.a.service' 2024-03-26T18:18:36.733 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:18:36.733 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-26T18:18:36.733 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-26T18:18:36.733 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.b 2024-03-26T18:18:37.229 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:36 smithi115 systemd[1]: Stopping Ceph mon.b for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:18:37.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:37 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-b[36170]: 2024-03-26T18:18:37.358+0000 7f56e2983700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:18:37.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:37 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-b[36170]: 2024-03-26T18:18:37.358+0000 7f56e2983700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-26T18:18:37.730 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:37 smithi115 podman[72680]: 2024-03-26 18:18:37.607935429 +0000 UTC m=+0.386188262 container died e80310a4705ba6b42f0b5770adf200a32a03e4b8948209e5637a3c8777308df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:18:43.717 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:43 smithi115 podman[72680]: 2024-03-26 18:18:43.458739392 +0000 UTC m=+6.236992226 container cleanup e80310a4705ba6b42f0b5770adf200a32a03e4b8948209e5637a3c8777308df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-26T18:18:43.717 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:43 smithi115 bash[72680]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-b 2024-03-26T18:18:43.979 INFO:journalctl@ceph.mon.b.smithi115.stdout:Mar 26 18:18:43 smithi115 podman[72691]: 2024-03-26 18:18:43.743741517 +0000 UTC m=+6.083804550 container remove e80310a4705ba6b42f0b5770adf200a32a03e4b8948209e5637a3c8777308df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True) 2024-03-26T18:18:43.999 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.b.service' 2024-03-26T18:18:44.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:18:44.061 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-26T18:18:44.062 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-26T18:18:44.062 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.c 2024-03-26T18:18:44.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:44 smithi171 systemd[1]: Stopping Ceph mon.c for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:18:44.838 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:44 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-c[36007]: 2024-03-26T18:18:44.512+0000 7f1bd431b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:18:44.838 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:44 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-c[36007]: 2024-03-26T18:18:44.512+0000 7f1bd431b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-26T18:18:44.838 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:44 smithi171 podman[73488]: 2024-03-26 18:18:44.575023317 +0000 UTC m=+0.220390205 container died 4f0304e95a283bf5066bea295d4391dcf0c7e83dafb1a699c8bd2286ef094558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-26T18:18:45.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:44 smithi171 podman[73488]: 2024-03-26 18:18:44.963651295 +0000 UTC m=+0.609018176 container cleanup 4f0304e95a283bf5066bea295d4391dcf0c7e83dafb1a699c8bd2286ef094558 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-26T18:18:45.098 INFO:journalctl@ceph.mon.c.smithi171.stdout:Mar 26 18:18:44 smithi171 bash[73488]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mon-c 2024-03-26T18:18:45.356 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mon.c.service' 2024-03-26T18:18:45.417 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:18:45.417 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-26T18:18:45.417 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-26T18:18:45.417 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.x 2024-03-26T18:18:45.744 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:18:45 smithi049 systemd[1]: Stopping Ceph mgr.x for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:18:47.244 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:18:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x[32562]: 2024-03-26T18:18:46.777+0000 7f3530224700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:18:47.244 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:18:46 smithi049 podman[93126]: 2024-03-26 18:18:46.817886024 +0000 UTC m=+1.013391520 container died 819b241b3d514701469d7787a79d6180f4ea596c1d36e065aac1aeb5f1762f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-26T18:18:48.244 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:18:48 smithi049 podman[93126]: 2024-03-26 18:18:48.01909156 +0000 UTC m=+2.214597045 container cleanup 819b241b3d514701469d7787a79d6180f4ea596c1d36e065aac1aeb5f1762f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:18:48.244 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:18:48 smithi049 bash[93126]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x 2024-03-26T18:18:48.508 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-26T18:18:48.537 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:18:48 smithi049 podman[93141]: 2024-03-26 18:18:48.295996499 +0000 UTC m=+1.397478155 container remove 819b241b3d514701469d7787a79d6180f4ea596c1d36e065aac1aeb5f1762f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:18:48.537 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:18:48 smithi049 systemd[1]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.x.service: Deactivated successfully. 2024-03-26T18:18:48.537 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:18:48 smithi049 systemd[1]: Stopped Ceph mgr.x for 5a3632a4-eb99-11ee-95d0-87774f69a715. 2024-03-26T18:18:48.537 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Mar 26 18:18:48 smithi049 systemd[1]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.x.service: Consumed 1min 36.287s CPU time. 2024-03-26T18:18:48.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:18:48.551 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-26T18:18:48.551 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-26T18:18:48.551 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.y 2024-03-26T18:18:48.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:18:48 smithi115 systemd[1]: Stopping Ceph mgr.y for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:18:49.180 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:18:48 smithi115 podman[72812]: 2024-03-26 18:18:48.925552224 +0000 UTC m=+0.150937936 container died e3e1ac33f56396f3c649d282d674eeec7bf9113553c6a1c6763905b361ab141a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-26T18:18:49.479 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:18:49 smithi115 podman[72812]: 2024-03-26 18:18:49.330321995 +0000 UTC m=+0.555707705 container cleanup e3e1ac33f56396f3c649d282d674eeec7bf9113553c6a1c6763905b361ab141a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:18:49.479 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Mar 26 18:18:49 smithi115 bash[72812]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-y 2024-03-26T18:18:49.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-26T18:18:49.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:18:49.752 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-26T18:18:49.752 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-26T18:18:49.753 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.z 2024-03-26T18:18:50.090 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:18:49 smithi171 systemd[1]: Stopping Ceph mgr.z for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:18:50.348 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:18:50 smithi171 podman[73610]: 2024-03-26 18:18:50.116534699 +0000 UTC m=+0.138627498 container died ecd495246d5c9d34b16331e061c1c346bce3470162aa894e49c1c5759c77d723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-26T18:18:50.765 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:18:50 smithi171 podman[73610]: 2024-03-26 18:18:50.514039938 +0000 UTC m=+0.536132728 container cleanup ecd495246d5c9d34b16331e061c1c346bce3470162aa894e49c1c5759c77d723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-26T18:18:50.765 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Mar 26 18:18:50 smithi171 bash[73610]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mgr-z 2024-03-26T18:18:50.918 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mgr.z.service' 2024-03-26T18:18:50.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:18:50.955 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-26T18:18:50.955 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-26T18:18:50.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.0 2024-03-26T18:18:51.245 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:51 smithi049 systemd[1]: Stopping Ceph osd.0 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:18:51.744 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0[46715]: 2024-03-26T18:18:51.307+0000 7f81ff32f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:18:51.744 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0[46715]: 2024-03-26T18:18:51.307+0000 7f81ff32f700 -1 osd.0 119 *** Got signal Terminated *** 2024-03-26T18:18:51.744 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0[46715]: 2024-03-26T18:18:51.307+0000 7f81ff32f700 -1 osd.0 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:18:57.456 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:57 smithi049 podman[93250]: 2024-03-26 18:18:57.195901322 +0000 UTC m=+6.016480521 container died 0c21864abd9d3b36e7459ce82ec6d7504457f1b731bafe11cbf9a94c81e9f573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T18:18:57.744 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:57 smithi049 podman[93250]: 2024-03-26 18:18:57.564397398 +0000 UTC m=+6.384976604 container cleanup 0c21864abd9d3b36e7459ce82ec6d7504457f1b731bafe11cbf9a94c81e9f573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T18:18:57.744 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:57 smithi049 bash[93250]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0 2024-03-26T18:18:58.164 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:57 smithi049 podman[93269]: 2024-03-26 18:18:57.866223665 +0000 UTC m=+0.663176095 container remove 0c21864abd9d3b36e7459ce82ec6d7504457f1b731bafe11cbf9a94c81e9f573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-26T18:18:58.426 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:58 smithi049 podman[93360]: 2024-03-26 18:18:58.093892842 +0000 UTC m=+0.025463662 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:18:58.427 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:58 smithi049 podman[93360]: 2024-03-26 18:18:58.208884708 +0000 UTC m=+0.140455557 container create 1d7d8ef6ddf132184dbbc0a7e5915e6cb480036170607cef5bcadf071935838e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-26T18:18:58.744 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:58 smithi049 podman[93360]: 2024-03-26 18:18:58.453235697 +0000 UTC m=+0.384806507 container init 1d7d8ef6ddf132184dbbc0a7e5915e6cb480036170607cef5bcadf071935838e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T18:18:58.744 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:58 smithi049 podman[93360]: 2024-03-26 18:18:58.457215051 +0000 UTC m=+0.388785864 container start 1d7d8ef6ddf132184dbbc0a7e5915e6cb480036170607cef5bcadf071935838e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:18:58.744 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:18:58 smithi049 podman[93360]: 2024-03-26 18:18:58.595088937 +0000 UTC m=+0.526659755 container attach 1d7d8ef6ddf132184dbbc0a7e5915e6cb480036170607cef5bcadf071935838e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-26T18:19:00.664 INFO:journalctl@ceph.osd.0.smithi049.stdout:Mar 26 18:19:00 smithi049 podman[93360]: 2024-03-26 18:19:00.386767391 +0000 UTC m=+2.318338206 container died 1d7d8ef6ddf132184dbbc0a7e5915e6cb480036170607cef5bcadf071935838e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-26T18:19:01.085 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.0.service' 2024-03-26T18:19:01.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:19:01.138 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-26T18:19:01.138 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-26T18:19:01.138 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.1 2024-03-26T18:19:01.449 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:01 smithi115 systemd[1]: Stopping Ceph osd.1 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:19:01.729 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1[40773]: 2024-03-26T18:19:01.474+0000 7ff35e4b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:19:01.729 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1[40773]: 2024-03-26T18:19:01.474+0000 7ff35e4b0700 -1 osd.1 119 *** Got signal Terminated *** 2024-03-26T18:19:01.729 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1[40773]: 2024-03-26T18:19:01.474+0000 7ff35e4b0700 -1 osd.1 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:19:07.443 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:07 smithi115 podman[72936]: 2024-03-26 18:19:07.145404519 +0000 UTC m=+5.792797558 container died b4373cd2f114ad3961617849fd9bf9135d023616173472048dace52360a65c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:19:07.729 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:07 smithi115 podman[72936]: 2024-03-26 18:19:07.56757708 +0000 UTC m=+6.214970108 container cleanup b4373cd2f114ad3961617849fd9bf9135d023616173472048dace52360a65c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-26T18:19:07.729 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:07 smithi115 bash[72936]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1 2024-03-26T18:19:08.193 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:07 smithi115 podman[72954]: 2024-03-26 18:19:07.803952314 +0000 UTC m=+0.652265789 container remove b4373cd2f114ad3961617849fd9bf9135d023616173472048dace52360a65c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-26T18:19:08.456 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:08 smithi115 podman[73042]: 2024-03-26 18:19:08.122564208 +0000 UTC m=+0.017549020 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:19:08.456 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:08 smithi115 podman[73042]: 2024-03-26 18:19:08.23025761 +0000 UTC m=+0.125242400 container create 6b751b0405c5d147e44abd29a1192edc9df7b88d4fb9bba2fedc974b96ccf866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-26T18:19:08.456 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:08 smithi115 podman[73042]: 2024-03-26 18:19:08.482856841 +0000 UTC m=+0.377841641 container init 6b751b0405c5d147e44abd29a1192edc9df7b88d4fb9bba2fedc974b96ccf866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-26T18:19:08.729 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:08 smithi115 podman[73042]: 2024-03-26 18:19:08.486174681 +0000 UTC m=+0.381159479 container start 6b751b0405c5d147e44abd29a1192edc9df7b88d4fb9bba2fedc974b96ccf866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-26T18:19:08.729 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:08 smithi115 podman[73042]: 2024-03-26 18:19:08.641653148 +0000 UTC m=+0.536638042 container attach 6b751b0405c5d147e44abd29a1192edc9df7b88d4fb9bba2fedc974b96ccf866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-26T18:19:09.543 INFO:journalctl@ceph.osd.1.smithi115.stdout:Mar 26 18:19:09 smithi115 podman[73042]: 2024-03-26 18:19:09.30545332 +0000 UTC m=+1.200438126 container died 6b751b0405c5d147e44abd29a1192edc9df7b88d4fb9bba2fedc974b96ccf866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-26T18:19:09.956 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.1.service' 2024-03-26T18:19:09.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:19:09.998 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-26T18:19:09.998 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-26T18:19:09.998 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.2 2024-03-26T18:19:10.348 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:10 smithi171 systemd[1]: Stopping Ceph osd.2 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:19:10.848 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:10 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2[41186]: 2024-03-26T18:19:10.469+0000 7f5561405700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:19:10.848 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:10 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2[41186]: 2024-03-26T18:19:10.469+0000 7f5561405700 -1 osd.2 119 *** Got signal Terminated *** 2024-03-26T18:19:10.848 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:10 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2[41186]: 2024-03-26T18:19:10.469+0000 7f5561405700 -1 osd.2 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:19:16.598 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:16 smithi171 podman[73748]: 2024-03-26 18:19:16.261932472 +0000 UTC m=+6.031890911 container died b5605db6d644e271f5dbaa379f6d32488b660495ddc04c51acb3ebfb16bf1517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-26T18:19:16.912 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:16 smithi171 podman[73748]: 2024-03-26 18:19:16.737133287 +0000 UTC m=+6.507091720 container cleanup b5605db6d644e271f5dbaa379f6d32488b660495ddc04c51acb3ebfb16bf1517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True) 2024-03-26T18:19:16.913 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:16 smithi171 bash[73748]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2 2024-03-26T18:19:17.319 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:16 smithi171 podman[73765]: 2024-03-26 18:19:16.93951441 +0000 UTC m=+0.604066662 container remove b5605db6d644e271f5dbaa379f6d32488b660495ddc04c51acb3ebfb16bf1517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-26T18:19:17.580 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:17 smithi171 podman[73854]: 2024-03-26 18:19:17.248509813 +0000 UTC m=+0.017966580 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:19:17.580 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:17 smithi171 podman[73854]: 2024-03-26 18:19:17.348952148 +0000 UTC m=+0.118408907 container create 262a0efa4792024470c09d76e3df8e4762d5f74a004312e219dacaf886ac2e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-26T18:19:17.580 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:17 smithi171 podman[73854]: 2024-03-26 18:19:17.606349675 +0000 UTC m=+0.375806442 container init 262a0efa4792024470c09d76e3df8e4762d5f74a004312e219dacaf886ac2e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-26T18:19:17.713 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:17 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:17.377+0000 7f0096b5c700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.896940+0000 front 2024-03-26T18:18:54.896975+0000 (oldest deadline 2024-03-26T18:19:17.197150+0000) 2024-03-26T18:19:17.848 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:17 smithi171 podman[73854]: 2024-03-26 18:19:17.610079579 +0000 UTC m=+0.379536329 container start 262a0efa4792024470c09d76e3df8e4762d5f74a004312e219dacaf886ac2e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-26T18:19:17.848 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:17 smithi171 podman[73854]: 2024-03-26 18:19:17.728281616 +0000 UTC m=+0.497738385 container attach 262a0efa4792024470c09d76e3df8e4762d5f74a004312e219dacaf886ac2e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-26T18:19:17.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:17 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:17.733+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:18.598 INFO:journalctl@ceph.osd.2.smithi171.stdout:Mar 26 18:19:18 smithi171 podman[73854]: 2024-03-26 18:19:18.360231783 +0000 UTC m=+1.129688543 container died 262a0efa4792024470c09d76e3df8e4762d5f74a004312e219dacaf886ac2e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T18:19:18.744 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:18 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:18.375+0000 7f0096b5c700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.896940+0000 front 2024-03-26T18:18:54.896975+0000 (oldest deadline 2024-03-26T18:19:17.197150+0000) 2024-03-26T18:19:18.973 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.2.service' 2024-03-26T18:19:18.978 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:18.706+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:19.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:19:19.017 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-26T18:19:19.017 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-26T18:19:19.017 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.3 2024-03-26T18:19:19.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:18 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:18.879+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:19.357 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:19 smithi049 systemd[1]: Stopping Ceph osd.3 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:19:19.716 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:19 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:19.383+0000 7f0096b5c700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.896940+0000 front 2024-03-26T18:18:54.896975+0000 (oldest deadline 2024-03-26T18:19:17.197150+0000) 2024-03-26T18:19:19.716 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:19 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:19.393+0000 7f009e570700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:19:19.716 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:19 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:19.393+0000 7f009e570700 -1 osd.3 119 *** Got signal Terminated *** 2024-03-26T18:19:19.716 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:19 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:19.393+0000 7f009e570700 -1 osd.3 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:19:19.716 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:19 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:19.612+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:19.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:19 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:19.676+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:20.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:19 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:19.875+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:20.620 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:20.364+0000 7f0096b5c700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.896940+0000 front 2024-03-26T18:18:54.896975+0000 (oldest deadline 2024-03-26T18:19:17.197150+0000) 2024-03-26T18:19:20.902 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:20.646+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:20.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:20 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:20.718+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:21.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:20 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:20.864+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:21.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:20 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:20.679+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:21.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:20 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:20.928+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:21.666 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:21 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:21.347+0000 7f0096b5c700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.896940+0000 front 2024-03-26T18:18:54.896975+0000 (oldest deadline 2024-03-26T18:19:17.197150+0000) 2024-03-26T18:19:21.942 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:21 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:21.692+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:21.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:21 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:21.696+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:22.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:21 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:21.852+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:22.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:21 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:21.663+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:22.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:21 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:21.968+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:22.680 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:22 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:22.312+0000 7f0096b5c700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.896940+0000 front 2024-03-26T18:18:54.896975+0000 (oldest deadline 2024-03-26T18:19:17.197150+0000) 2024-03-26T18:19:22.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:22 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:22.653+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:22.988 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:22 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:22.706+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:23.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:22 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:22.708+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:23.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:22 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:22.854+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:23.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:23 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:23.014+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:23.691 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:23 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:23.326+0000 7f0096b5c700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.896940+0000 front 2024-03-26T18:18:54.896975+0000 (oldest deadline 2024-03-26T18:19:17.197150+0000) 2024-03-26T18:19:23.973 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:23 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:23.717+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:23.978 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:23 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:23.631+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:24.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:23 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:23.713+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:24.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:23 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:23.884+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:24.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:23 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:23.999+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:24.702 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:24 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3[53252]: 2024-03-26T18:19:24.337+0000 7f0096b5c700 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.896940+0000 front 2024-03-26T18:18:54.896975+0000 (oldest deadline 2024-03-26T18:19:17.197150+0000) 2024-03-26T18:19:24.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:24 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:24.645+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:24.990 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:24 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:24.728+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:25.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:24.723+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:25.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:24.837+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:25.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:25 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:25.016+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:25.244 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:25 smithi049 podman[93476]: 2024-03-26 18:19:25.075403263 +0000 UTC m=+5.812965399 container died 69e58e37b585cb2aa3656df605480e2b40ff036d0b7b3bf56d425d9012fb8fb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:19:25.660 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:25 smithi049 podman[93476]: 2024-03-26 18:19:25.485600042 +0000 UTC m=+6.223162177 container cleanup 69e58e37b585cb2aa3656df605480e2b40ff036d0b7b3bf56d425d9012fb8fb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:19:25.661 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:25 smithi049 bash[93476]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3 2024-03-26T18:19:25.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:25 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:25.673+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:25.994 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:25 smithi049 podman[93494]: 2024-03-26 18:19:25.687161756 +0000 UTC m=+0.603928944 container remove 69e58e37b585cb2aa3656df605480e2b40ff036d0b7b3bf56d425d9012fb8fb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:19:25.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:25 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:25.753+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:26.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:25 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:25.690+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:26.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:25 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:25.796+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:26.348 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:26 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:26.053+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:26.348 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:26 smithi049 podman[93586]: 2024-03-26 18:19:26.017841121 +0000 UTC m=+0.018360249 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:19:26.348 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:26 smithi049 podman[93586]: 2024-03-26 18:19:26.154798075 +0000 UTC m=+0.155317170 container create 631509f1529156a7b0c5bfeed854a42a2771de974b0a08b689c37bd71c3e0db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:19:26.659 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:26 smithi049 podman[93586]: 2024-03-26 18:19:26.374796452 +0000 UTC m=+0.375315549 container init 631509f1529156a7b0c5bfeed854a42a2771de974b0a08b689c37bd71c3e0db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-26T18:19:26.659 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:26 smithi049 podman[93586]: 2024-03-26 18:19:26.378360506 +0000 UTC m=+0.378879602 container start 631509f1529156a7b0c5bfeed854a42a2771de974b0a08b689c37bd71c3e0db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-26T18:19:26.659 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:26 smithi049 podman[93586]: 2024-03-26 18:19:26.466059344 +0000 UTC m=+0.466578447 container attach 631509f1529156a7b0c5bfeed854a42a2771de974b0a08b689c37bd71c3e0db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-26T18:19:26.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:26 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:26.722+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:26.994 INFO:journalctl@ceph.osd.3.smithi049.stdout:Mar 26 18:19:26 smithi049 podman[93586]: 2024-03-26 18:19:26.686074644 +0000 UTC m=+0.686593748 container died 631509f1529156a7b0c5bfeed854a42a2771de974b0a08b689c37bd71c3e0db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True) 2024-03-26T18:19:26.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:26 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:26.749+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:26.999 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:26.736+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:26.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:26.750+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:27.377 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:27 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:27.049+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:27.402 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.3.service' 2024-03-26T18:19:27.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:19:27.454 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-26T18:19:27.454 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-26T18:19:27.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.4 2024-03-26T18:19:27.845 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:27 smithi115 systemd[1]: Stopping Ceph osd.4 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:19:27.845 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:27 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:27.738+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:27.846 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:27 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:27.738+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:28.017 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:27 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:27.774+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:28.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:27 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:27.743+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:28.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:27 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:27.707+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:28.230 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:27 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4[47289]: 2024-03-26T18:19:27.871+0000 7fe1c0839700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:19:28.231 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:27 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4[47289]: 2024-03-26T18:19:27.871+0000 7fe1c0839700 -1 osd.4 119 *** Got signal Terminated *** 2024-03-26T18:19:28.231 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:27 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4[47289]: 2024-03-26T18:19:27.871+0000 7fe1c0839700 -1 osd.4 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:19:28.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:28 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:28.044+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:28.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:28 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:28.742+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:29.012 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:28 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:28.742+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:29.012 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:28 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:28.766+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:29.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:28 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:28.710+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:29.099 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:28 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:28.715+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:29.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:29 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:29.081+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:29.979 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:29 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:29.781+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:29.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:29 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:29.732+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:29.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:29 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:29.732+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:29.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:29 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:29.759+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:29.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:29 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:29.759+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:30.089 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:29 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:29.753+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:30.090 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:29 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:29.731+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:30.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:30.116+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:30.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:30.065+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:30.979 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4[47289]: 2024-03-26T18:19:30.634+0000 7fe1bce2d700 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.644024+0000 front 2024-03-26T18:19:04.644064+0000 (oldest deadline 2024-03-26T18:19:30.544323+0000) 2024-03-26T18:19:30.979 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:30.740+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:30.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:30.736+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:30.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:30 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:30.736+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:30.993 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:30.753+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:30.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:30 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:30.753+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:31.055 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:30.742+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:31.056 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:30.714+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:31.056 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:30.714+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:31.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:31 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:31.082+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:31.493 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:31 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:31.020+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:31.979 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:31 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4[47289]: 2024-03-26T18:19:31.642+0000 7fe1bce2d700 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.644024+0000 front 2024-03-26T18:19:04.644064+0000 (oldest deadline 2024-03-26T18:19:30.544323+0000) 2024-03-26T18:19:31.979 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:31 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:31.740+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:31.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:31 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:31.757+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:31.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:31 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:31.757+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:31.993 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:31 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:31.710+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:31.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:31 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:31.710+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:32.059 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:31 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:31.700+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:32.059 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:31 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:31.758+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:32.059 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:31 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:31.758+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:32.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:32 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:32.085+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:32.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:32 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:32.058+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:32.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:32 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:32.058+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:32.979 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:32 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4[47289]: 2024-03-26T18:19:32.605+0000 7fe1bce2d700 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.644024+0000 front 2024-03-26T18:19:04.644064+0000 (oldest deadline 2024-03-26T18:19:30.544323+0000) 2024-03-26T18:19:32.979 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:32 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:32.729+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:32.980 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:32 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:32.736+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:32.980 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:32 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:32.736+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:32.993 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:32 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:32.721+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:32.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:32 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:32.721+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:33.033 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:32 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:32.734+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:33.033 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:32 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:32.734+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:33.033 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:32 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:32.797+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:33.033 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:32 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:32.797+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:33.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:33 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:33.059+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:33.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:33 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:33.038+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:33.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:33 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:33.038+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:33.921 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:33 smithi115 podman[73159]: 2024-03-26 18:19:33.669899241 +0000 UTC m=+5.998003840 container died 6078ddadc0756c9ecd681faadb6da627e3a90fd2b625d0ca0cd99c093067f774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-26T18:19:33.921 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:33 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:33.714+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:33.922 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:33 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:33.745+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:33.922 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:33 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:33.745+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:33.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:33 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:33.734+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:33.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:33 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:33.734+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:34.035 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:33 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:33.813+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:34.035 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:33 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:33.813+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:34.036 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:33 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:33.711+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:34.036 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:33 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:33.711+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:34.228 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:34 smithi115 podman[73159]: 2024-03-26 18:19:34.038434478 +0000 UTC m=+6.366539068 container cleanup 6078ddadc0756c9ecd681faadb6da627e3a90fd2b625d0ca0cd99c093067f774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T18:19:34.229 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:34 smithi115 bash[73159]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4 2024-03-26T18:19:34.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:34 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:34.061+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:34.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:34 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:34.070+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:34.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:34 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:34.070+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:34.619 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:34 smithi115 podman[73177]: 2024-03-26 18:19:34.307435826 +0000 UTC m=+0.635198642 container remove 6078ddadc0756c9ecd681faadb6da627e3a90fd2b625d0ca0cd99c093067f774 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0) 2024-03-26T18:19:34.892 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:34 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:34.698+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:34.892 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:34 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:34.761+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:34.892 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:34 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:34.761+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:34.893 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:34 smithi115 podman[73271]: 2024-03-26 18:19:34.549696424 +0000 UTC m=+0.018450055 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:19:34.893 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:34 smithi115 podman[73271]: 2024-03-26 18:19:34.666529128 +0000 UTC m=+0.135282755 container create ecf58840c9225dcf98f855376f5be5fed7dbed6616859b1de6a8995945022eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-26T18:19:34.989 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:34 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:34.766+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:34.989 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:34 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:34.766+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:34.990 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:34 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:34.750+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:34.990 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:34 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:34.750+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:34.993 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:34 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:34.716+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:34.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:34 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:34.716+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:35.183 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:34 smithi115 podman[73271]: 2024-03-26 18:19:34.919064849 +0000 UTC m=+0.387818483 container init ecf58840c9225dcf98f855376f5be5fed7dbed6616859b1de6a8995945022eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2) 2024-03-26T18:19:35.183 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:34 smithi115 podman[73271]: 2024-03-26 18:19:34.922212424 +0000 UTC m=+0.390966050 container start ecf58840c9225dcf98f855376f5be5fed7dbed6616859b1de6a8995945022eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-26T18:19:35.183 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:35 smithi115 podman[73271]: 2024-03-26 18:19:35.002176037 +0000 UTC m=+0.470929676 container attach ecf58840c9225dcf98f855376f5be5fed7dbed6616859b1de6a8995945022eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-03-26T18:19:35.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:35 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:35.015+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:35.478 INFO:journalctl@ceph.osd.4.smithi115.stdout:Mar 26 18:19:35 smithi115 podman[73271]: 2024-03-26 18:19:35.209698775 +0000 UTC m=+0.678452402 container died ecf58840c9225dcf98f855376f5be5fed7dbed6616859b1de6a8995945022eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-26T18:19:35.493 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:35 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:35.078+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:35.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:35 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:35.078+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:35.805 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:35 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:35.725+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:35.805 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:35 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:35.753+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:35.805 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:35 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:35.753+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:35.824 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.4.service' 2024-03-26T18:19:35.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:19:35.869 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-26T18:19:35.869 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-26T18:19:35.869 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.5 2024-03-26T18:19:35.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:35 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:35.701+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:35.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:35 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:35.701+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:36.034 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:35 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:35.763+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:36.034 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:35 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:35.763+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:36.034 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:35 smithi171 systemd[1]: Stopping Ceph osd.5 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:19:36.034 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:35 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:35.785+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:36.035 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:35 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:35.785+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:36.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:36 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:36.059+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:36.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:36 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:36.087+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:36.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:36 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:36.087+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:36.747 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:36 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:36.437+0000 7f856fb90700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:19:36.748 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:36 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:36.437+0000 7f856fb90700 -1 osd.5 119 *** Got signal Terminated *** 2024-03-26T18:19:36.748 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:36 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:36.437+0000 7f856fb90700 -1 osd.5 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:19:36.979 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:36 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:36.723+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:36.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:36 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:36.760+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:36.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:36 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:36.760+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:36.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:36 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:36.670+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:36.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:36 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:36.670+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:37.027 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:36 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:36.800+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:37.028 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:36 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:36.800+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:37.028 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:36 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:36.774+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:37.028 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:36 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:36.774+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:37.028 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:36 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:36.774+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:37.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:37 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:37.052+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:37.452 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:37.126+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:37.452 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:37.126+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:37.744 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:37.623+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:37.744 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:37 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:37.623+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:37.979 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:37 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:37.748+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:37.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:37 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:37.755+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:37.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:37 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:37.755+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:38.070 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:37 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:37.763+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:38.070 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:37 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:37.763+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:38.070 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:37 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:37.772+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:38.070 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:37 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:37.772+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:38.070 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:37 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:37.772+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:38.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:38 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:38.095+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:38.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:38 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:38.099+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:38.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:38 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:38.099+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:38.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:38 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:38.099+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:38.979 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:38 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:38.784+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:38.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:38 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:38.716+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:38.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:38 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:38.716+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:38.993 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:38 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:38.622+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:38.993 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:38 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:38.622+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:39.091 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:38 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:38.773+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:39.091 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:38 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:38.773+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:39.091 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:38 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:38.773+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:39.091 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:38 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:38.729+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:39.091 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:38 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:38.729+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:39.092 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:38 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:38.730+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.159163+0000 front 2024-03-26T18:19:15.159189+0000 (oldest deadline 2024-03-26T18:19:38.059332+0000) 2024-03-26T18:19:39.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:39 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:39.115+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:39.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:39.052+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:39.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:39.052+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:39.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:39.052+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:39.979 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:39 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:39.813+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:39.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:39 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:39.706+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:39.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:39 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:39.706+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:39.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:39 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:39.706+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:39.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:39.630+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:39.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:39.630+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:40.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:39 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:39.762+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:40.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:39 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:39.762+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:40.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:39 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:39.762+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.159163+0000 front 2024-03-26T18:19:15.159189+0000 (oldest deadline 2024-03-26T18:19:38.059332+0000) 2024-03-26T18:19:40.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:39 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:39.821+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:40.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:39 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:39.821+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:40.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:39 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:39.821+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:40.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:40 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:40.067+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:40.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:40 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:40.067+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:40.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:40 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:40.067+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:40.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:40 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:40.135+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:40.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:40 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:40.135+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:40.979 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:40 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:40.833+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:40.979 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:40 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:40.833+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:40.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:40 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:40.735+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:40.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:40 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:40.735+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:40.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:40 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:40.735+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:40.993 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:40 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:40.607+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:40.993 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:40 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:40.607+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:41.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:40 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:40.777+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:53.655263+0000 front 2024-03-26T18:18:53.655217+0000 (oldest deadline 2024-03-26T18:19:20.655904+0000) 2024-03-26T18:19:41.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:40 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:40.777+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.257548+0000 front 2024-03-26T18:19:05.257567+0000 (oldest deadline 2024-03-26T18:19:29.957535+0000) 2024-03-26T18:19:41.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:40 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5[47726]: 2024-03-26T18:19:40.777+0000 7f856c184700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.159163+0000 front 2024-03-26T18:19:15.159189+0000 (oldest deadline 2024-03-26T18:19:38.059332+0000) 2024-03-26T18:19:41.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:40 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:40.845+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:41.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:40 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:40.845+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:41.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:40 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:40.845+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:41.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:41 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:41.101+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:41.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:41 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:41.101+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:41.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:41 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:41.101+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:41.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:41 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:41.152+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:41.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:41 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:41.152+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:41.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:41 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:41.571+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:41.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:41 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:41.571+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:41.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:41 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:41.571+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.100937+0000 front 2024-03-26T18:19:15.100895+0000 (oldest deadline 2024-03-26T18:19:40.901373+0000) 2024-03-26T18:19:42.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:41 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:41.836+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:42.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:41 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:41.836+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:42.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:41 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:41.836+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:42.228 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:41 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:41.870+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:42.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:41 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:41.870+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:42.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:41 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:41.759+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:42.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:41 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:41.759+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:42.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:41 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:41.759+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:42.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:42 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:42.067+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:42.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:42 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:42.067+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:42.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:42 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:42.067+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:42.571 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:42 smithi171 podman[73971]: 2024-03-26 18:19:42.295039997 +0000 UTC m=+6.207671052 container died e6a09bd763d13d4141dd4d3951550f4d8d23345a546f2344b014affd193b4d68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-26T18:19:42.572 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:42 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:42.160+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:42.572 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:42 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:42.160+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:42.841 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:42 smithi171 podman[73971]: 2024-03-26 18:19:42.69745634 +0000 UTC m=+6.610087395 container cleanup e6a09bd763d13d4141dd4d3951550f4d8d23345a546f2344b014affd193b4d68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True) 2024-03-26T18:19:42.841 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:42 smithi171 bash[73971]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5 2024-03-26T18:19:42.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:42 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:42.532+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:42.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:42 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:42.532+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:42.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:42 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:42.532+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.100937+0000 front 2024-03-26T18:19:15.100895+0000 (oldest deadline 2024-03-26T18:19:40.901373+0000) 2024-03-26T18:19:43.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:42 smithi171 podman[73988]: 2024-03-26 18:19:42.966320622 +0000 UTC m=+0.655139269 container remove e6a09bd763d13d4141dd4d3951550f4d8d23345a546f2344b014affd193b4d68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-26T18:19:43.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:42 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:42.865+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:43.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:42 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:42.865+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:43.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:42 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:42.865+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:43.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:42 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:42.916+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:43.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:42 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:42.916+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:43.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:42 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:42.796+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:43.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:42 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:42.796+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:43.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:42 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:42.796+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:43.399 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:43 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:43.142+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:43.399 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:43 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:43.142+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:43.399 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:43 smithi171 podman[74081]: 2024-03-26 18:19:43.174309885 +0000 UTC m=+0.018138879 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:19:43.399 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:43 smithi171 podman[74081]: 2024-03-26 18:19:43.309670683 +0000 UTC m=+0.153499663 container create c9044d077fdfc1f03bda03b577dc2781f1e9f875b2d0c062ee379ffdf448d4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T18:19:43.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:43 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:43.076+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:43.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:43 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:43.076+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:43.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:43 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:43.076+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:43.800 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:43 smithi171 podman[74081]: 2024-03-26 18:19:43.538136024 +0000 UTC m=+0.381965008 container init c9044d077fdfc1f03bda03b577dc2781f1e9f875b2d0c062ee379ffdf448d4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-26T18:19:43.800 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:43 smithi171 podman[74081]: 2024-03-26 18:19:43.541600014 +0000 UTC m=+0.385428994 container start c9044d077fdfc1f03bda03b577dc2781f1e9f875b2d0c062ee379ffdf448d4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True) 2024-03-26T18:19:43.800 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:43 smithi171 podman[74081]: 2024-03-26 18:19:43.596075326 +0000 UTC m=+0.439904322 container attach c9044d077fdfc1f03bda03b577dc2781f1e9f875b2d0c062ee379ffdf448d4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:19:43.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:43 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:43.533+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:43.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:43 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:43.533+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:43.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:43 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:43.533+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.100937+0000 front 2024-03-26T18:19:15.100895+0000 (oldest deadline 2024-03-26T18:19:40.901373+0000) 2024-03-26T18:19:44.098 INFO:journalctl@ceph.osd.5.smithi171.stdout:Mar 26 18:19:43 smithi171 podman[74081]: 2024-03-26 18:19:43.827526341 +0000 UTC m=+0.671355325 container died c9044d077fdfc1f03bda03b577dc2781f1e9f875b2d0c062ee379ffdf448d4d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-26T18:19:44.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:43 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:43.870+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:44.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:43 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:43.870+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:44.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:43 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:43.870+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:44.228 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:43 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:43.926+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:44.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:43 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:43.926+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:44.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:43 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:43.837+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:44.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:43 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:43.837+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:44.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:43 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:43.837+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:44.375 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:44 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:44.125+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:44.375 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:44 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:44.125+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:44.400 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.5.service' 2024-03-26T18:19:44.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:19:44.443 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-26T18:19:44.444 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-26T18:19:44.444 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.6 2024-03-26T18:19:44.477 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:44 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:44.033+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:44.477 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:44 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:44.033+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:44.477 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:44 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:44.033+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:44.744 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:44 smithi049 systemd[1]: Stopping Ceph osd.6 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:19:44.744 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:44 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:44.581+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:44.744 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:44 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:44.581+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:44.744 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:44 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:44.581+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.100937+0000 front 2024-03-26T18:19:15.100895+0000 (oldest deadline 2024-03-26T18:19:40.901373+0000) 2024-03-26T18:19:45.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:44 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:44.959+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:45.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:44 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:44.959+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:45.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:44 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:44.838+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:45.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:44 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:44.838+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:45.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:44 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:44.838+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:45.243 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:44 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:44.797+0000 7fdeb5fa8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:19:45.244 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:44 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:44.797+0000 7fdeb5fa8700 -1 osd.6 119 *** Got signal Terminated *** 2024-03-26T18:19:45.244 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:44 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:44.797+0000 7fdeb5fa8700 -1 osd.6 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:19:45.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:45 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:45.019+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:45.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:45 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:45.019+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:45.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:45 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:45.019+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:45.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:45 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:45.082+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:45.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:45 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:45.082+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:45.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:44 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:44.904+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:45.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:44 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:44.904+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:45.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:44 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:44.904+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:45.988 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:45 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:45.610+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:45.989 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:45 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:45.610+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:45.989 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:45 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:45.610+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.100937+0000 front 2024-03-26T18:19:15.100895+0000 (oldest deadline 2024-03-26T18:19:40.901373+0000) 2024-03-26T18:19:46.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:45 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:45.937+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:46.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:45 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:45.937+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:46.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:45 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:45.791+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:46.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:45 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:45.791+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:46.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:45 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:45.791+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:46.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:46.014+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:46.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:46.014+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:46.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:46.014+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:46.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:45 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:45.942+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:46.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:45 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:45.942+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:46.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:45 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:45.942+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:46.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:46 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:46.067+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:46.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:46 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:46.067+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:46.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:46 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:46.067+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:46.961 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:46.612+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:46.961 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:46.612+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:46.961 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:46.612+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.100937+0000 front 2024-03-26T18:19:15.100895+0000 (oldest deadline 2024-03-26T18:19:40.901373+0000) 2024-03-26T18:19:46.961 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:46.612+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.901844+0000 front 2024-03-26T18:19:20.901816+0000 (oldest deadline 2024-03-26T18:19:46.201969+0000) 2024-03-26T18:19:47.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:46 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:46.927+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:47.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:46 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:46.927+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:47.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:46 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:46.927+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:47.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:46 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:46.838+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:47.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:46 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:46.838+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:47.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:46 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:46.838+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:47.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:46 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:46.838+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:47.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:46.987+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:47.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:46.987+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:47.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:46.987+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:47.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:47 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:47.019+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:47.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:47 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:47.019+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:47.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:47 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:47.019+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:47.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:46 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:46.907+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:47.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:46 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:46.907+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:47.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:46 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:46.907+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:47.947 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:47 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:47.572+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:47.947 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:47 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:47.572+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:47.947 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:47 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:47.572+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.100937+0000 front 2024-03-26T18:19:15.100895+0000 (oldest deadline 2024-03-26T18:19:40.901373+0000) 2024-03-26T18:19:47.947 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:47 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:47.572+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.901844+0000 front 2024-03-26T18:19:20.901816+0000 (oldest deadline 2024-03-26T18:19:46.201969+0000) 2024-03-26T18:19:48.130 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:47 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:47.926+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:48.131 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:47 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:47.926+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:48.131 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:47 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:47.926+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:48.131 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:47 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:47.821+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:48.131 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:47 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:47.821+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:48.131 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:47 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:47.821+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:48.131 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:47 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:47.821+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:48.233 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:47 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:47.973+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:48.233 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:47 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:47.973+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:48.233 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:47 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:47.973+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:48.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:48 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:48.003+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:48.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:48 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:48.003+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:48.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:48 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:48.003+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:48.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:47 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:47.927+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:48.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:47 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:47.927+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:48.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:47 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:47.927+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:48.910 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:48 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:48.524+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:48.910 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:48 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:48.524+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:48.910 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:48 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:48.524+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.100937+0000 front 2024-03-26T18:19:15.100895+0000 (oldest deadline 2024-03-26T18:19:40.901373+0000) 2024-03-26T18:19:48.910 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:48 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:48.524+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.901844+0000 front 2024-03-26T18:19:20.901816+0000 (oldest deadline 2024-03-26T18:19:46.201969+0000) 2024-03-26T18:19:49.144 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:48 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:48.957+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:49.144 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:48 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:48.957+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:49.144 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:48 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:48.957+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:49.144 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:48 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:48.781+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:49.145 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:48 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:48.781+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:49.145 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:48 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:48.781+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:49.145 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:48 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:48.781+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:49.211 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:48 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:48.936+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:49.212 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:48 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:48.936+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:49.212 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:48 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:48.936+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:49.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:48 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:48.948+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:49.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:48 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:48.948+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:49.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:48 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:48.948+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:49.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:49 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:49.002+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:49.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:49 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:49.002+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:49.349 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:49 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:49.002+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:49.744 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:49 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:49.519+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.696325+0000 front 2024-03-26T18:18:54.696357+0000 (oldest deadline 2024-03-26T18:19:18.796542+0000) 2024-03-26T18:19:49.745 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:49 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:49.519+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.099401+0000 front 2024-03-26T18:19:05.099298+0000 (oldest deadline 2024-03-26T18:19:29.199313+0000) 2024-03-26T18:19:49.745 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:49 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:49.519+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:15.100937+0000 front 2024-03-26T18:19:15.100895+0000 (oldest deadline 2024-03-26T18:19:40.901373+0000) 2024-03-26T18:19:49.745 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:49 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6[60253]: 2024-03-26T18:19:49.519+0000 7fdeb259c700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.901844+0000 front 2024-03-26T18:19:20.901816+0000 (oldest deadline 2024-03-26T18:19:46.201969+0000) 2024-03-26T18:19:50.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:50 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:50.004+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:50.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:50 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:50.004+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:50.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:50 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:50.004+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:50.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:49 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:49.830+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:50.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:49 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:49.830+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:50.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:49 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:49.830+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:50.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:49 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:49.830+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:50.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:49 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:49.978+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:50.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:49 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:49.979+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:50.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:49 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:49.979+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:50.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:50 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:50.035+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:50.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:50 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:50.035+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:50.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:50 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:50.035+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:50.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:49 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:49.937+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:50.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:49 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:49.937+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:50.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:49 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:49.937+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:50.994 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:50 smithi049 podman[93691]: 2024-03-26 18:19:50.765019927 +0000 UTC m=+6.086662256 container died 00681a64db7ec8b8ef8881b13128c140889f68f3d55573c571b70e3f67579bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-26T18:19:50.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:50 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:50.949+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:50.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:50 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:50.949+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:50.995 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:50 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:50.949+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:50.995 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:50 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:50.949+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:19:51.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:50 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:50.875+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:51.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:50 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:50.875+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:51.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:50 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:50.875+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:51.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:50 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:50.875+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:51.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:51 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:51.009+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:51.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:51 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:51.009+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:51.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:51 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:51.009+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:51.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:51 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:51.040+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:51.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:51 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:51.040+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:51.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:51 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:51.040+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:51.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:50 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:50.945+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:51.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:50 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:50.945+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:51.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:50 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:50.945+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:51.390 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:51 smithi049 podman[93691]: 2024-03-26 18:19:51.173011013 +0000 UTC m=+6.494653345 container cleanup 00681a64db7ec8b8ef8881b13128c140889f68f3d55573c571b70e3f67579bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS) 2024-03-26T18:19:51.390 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:51 smithi049 bash[93691]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6 2024-03-26T18:19:51.701 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:51 smithi049 podman[93720]: 2024-03-26 18:19:51.416391045 +0000 UTC m=+0.643383559 container remove 00681a64db7ec8b8ef8881b13128c140889f68f3d55573c571b70e3f67579bcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-26T18:19:51.956 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:51 smithi049 podman[93814]: 2024-03-26 18:19:51.631419687 +0000 UTC m=+0.018804836 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:19:51.956 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:51 smithi049 podman[93814]: 2024-03-26 18:19:51.858882295 +0000 UTC m=+0.246267431 container create 5dc2cb87e1df466ac9cc828f2055f0fa9b554010a0013817900e81b4f24c37c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T18:19:51.956 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:51.939+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:51.956 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:51.939+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:51.956 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:51.939+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:51.956 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:51 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:51.939+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:19:52.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:51 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:51.966+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:52.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:51 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:51.966+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:52.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:51 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:51.966+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:52.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:51 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:51.828+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:52.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:51 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:51.828+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:52.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:51 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:51.828+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:52.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:51 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:51.828+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:52.243 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:52 smithi049 podman[93814]: 2024-03-26 18:19:52.136774643 +0000 UTC m=+0.524159782 container init 5dc2cb87e1df466ac9cc828f2055f0fa9b554010a0013817900e81b4f24c37c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-26T18:19:52.243 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:52 smithi049 podman[93814]: 2024-03-26 18:19:52.14040989 +0000 UTC m=+0.527795027 container start 5dc2cb87e1df466ac9cc828f2055f0fa9b554010a0013817900e81b4f24c37c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-26T18:19:52.244 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:52 smithi049 podman[93814]: 2024-03-26 18:19:52.228146099 +0000 UTC m=+0.615531255 container attach 5dc2cb87e1df466ac9cc828f2055f0fa9b554010a0013817900e81b4f24c37c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:19:52.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:52 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:52.079+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:52.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:52 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:52.079+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:52.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:52 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:52.079+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:52.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:51 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:51.939+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:52.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:51 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:51.939+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:52.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:51 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:51.939+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:52.722 INFO:journalctl@ceph.osd.6.smithi049.stdout:Mar 26 18:19:52 smithi049 podman[93814]: 2024-03-26 18:19:52.437928994 +0000 UTC m=+0.825314121 container died 5dc2cb87e1df466ac9cc828f2055f0fa9b554010a0013817900e81b4f24c37c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:19:52.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:52 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:52.983+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:52.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:52 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:52.983+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:52.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:52 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:52.983+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:52.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:52 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:52.983+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:19:53.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.6.service' 2024-03-26T18:19:53.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:19:53.106 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-26T18:19:53.106 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-26T18:19:53.106 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.7 2024-03-26T18:19:53.137 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:52 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:52.940+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:53.137 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:52 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:52.940+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:53.137 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:52 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:52.940+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:53.137 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:52 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:52.875+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:53.138 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:52 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:52.875+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:53.138 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:52 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:52.875+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:53.138 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:52 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:52.875+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:53.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:53 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:53.060+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:53.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:53 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:53.060+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:53.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:53 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:53.060+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:53.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:52 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:52.982+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:53.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:52 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:52.982+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:53.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:52 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:52.982+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:53.478 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:53 smithi115 systemd[1]: Stopping Ceph osd.7 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:19:53.941 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:53 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:53.616+0000 7f07241f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:19:53.941 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:53 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:53.616+0000 7f07241f5700 -1 osd.7 119 *** Got signal Terminated *** 2024-03-26T18:19:53.941 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:53 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:53.616+0000 7f07241f5700 -1 osd.7 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:19:53.942 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:53 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:53.849+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:53.942 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:53 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:53.849+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:53.942 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:53 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:53.849+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:53.942 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:53 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:53.849+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:54.228 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:53 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:53.967+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:54.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:53 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:53.967+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:54.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:53 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:53.967+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:54.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:53 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:53.952+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:54.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:53 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:53.952+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:54.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:53 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:53.952+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:54.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:53 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:53.952+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:19:54.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:54 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:54.066+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:54.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:54 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:54.066+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:54.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:54 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:54.066+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:54.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:53 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:53.973+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:54.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:53 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:53.973+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:54.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:53 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:53.973+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:54.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:53 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:53.973+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:19:55.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:54 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:54.936+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:55.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:54 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:54.936+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:55.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:54 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:54.936+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:55.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:54 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:54.853+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:55.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:54 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:54.853+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:55.230 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:54 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:54.853+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:55.230 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:54 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:54.853+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:55.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:54 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:54.970+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:55.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:54 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:54.970+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:55.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:54 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:54.970+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:55.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:54 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:54.970+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:19:55.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:54 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:54.970+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:19:55.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:55.112+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:55.349 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:55.112+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:55.349 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:55.112+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:55.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:54 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:54.969+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:55.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:54 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:54.969+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:55.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:54 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:54.969+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:55.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:54 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:54.969+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:19:56.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:55 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:55.882+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:56.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:55 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:55.882+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:56.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:55 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:55.882+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:56.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:55 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:55.882+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:56.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:55 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:55.957+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:56.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:55 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:55.957+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:56.230 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:55 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:55.957+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:56.230 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:55 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:55.957+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.053256+0000 front 2024-03-26T18:19:30.053507+0000 (oldest deadline 2024-03-26T18:19:55.953436+0000) 2024-03-26T18:19:56.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:55 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:55.949+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:56.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:55 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:55.949+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:56.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:55 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:55.949+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:56.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:55 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:55.949+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:19:56.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:55 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:55.949+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:19:56.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:55.920+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:56.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:55.920+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:56.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:55.920+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:56.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:55 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:55.920+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:19:56.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:56 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:56.122+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:56.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:56 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:56.122+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:56.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:56 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:56.122+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:56.349 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:56 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:56.122+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:19:57.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:56.994+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:57.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:56.994+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:57.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:56.994+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:57.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:56.994+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.053256+0000 front 2024-03-26T18:19:30.053507+0000 (oldest deadline 2024-03-26T18:19:55.953436+0000) 2024-03-26T18:19:57.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:56.923+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:57.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:56.923+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:57.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:56.923+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:57.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:56.923+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:57.230 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:56 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:56.923+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:19:57.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:56 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:56.997+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:57.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:56 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:56.997+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:57.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:56 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:56.997+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:57.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:56 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:56.997+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:19:57.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:56 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:56.997+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:19:57.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:57.153+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:57.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:57.153+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:57.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:57.153+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:57.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:57.153+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:19:57.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:56 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:56.907+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:57.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:56 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:56.907+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:57.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:56 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:56.907+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:57.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:56 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:56.907+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:19:58.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:57.953+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:00.947955+0000 front 2024-03-26T18:19:00.948021+0000 (oldest deadline 2024-03-26T18:19:29.748600+0000) 2024-03-26T18:19:58.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:57.953+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.649653+0000 front 2024-03-26T18:19:12.649634+0000 (oldest deadline 2024-03-26T18:19:40.250552+0000) 2024-03-26T18:19:58.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:57.953+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:21.951839+0000 front 2024-03-26T18:19:21.951867+0000 (oldest deadline 2024-03-26T18:19:46.651782+0000) 2024-03-26T18:19:58.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7[54171]: 2024-03-26T18:19:57.953+0000 7f071c7e1700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.053256+0000 front 2024-03-26T18:19:30.053507+0000 (oldest deadline 2024-03-26T18:19:55.953436+0000) 2024-03-26T18:19:58.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:57.963+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:58.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:57.963+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:58.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:57.963+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:58.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:57.963+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:58.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:57 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:57.963+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:19:58.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:57 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:57.955+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:58.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:57 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:57.955+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:58.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:57 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:57.955+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:58.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:57 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:57.955+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:19:58.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:57 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:57.955+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:19:58.269 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:58.167+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:58.270 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:58.167+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:58.270 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:58.167+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:58.270 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:58.167+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:19:58.270 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:57.953+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:58.270 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:57.953+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:58.270 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:57.953+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:58.270 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:57 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:57.953+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:19:58.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:58 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:58.942+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:19:58.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:58 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:58.942+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:19:58.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:58 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:58.942+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:19:58.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:58 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:58.942+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:19:58.994 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:58 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:58.942+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:19:59.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:58.979+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:59.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:58.979+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:59.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:58.979+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:59.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:58.979+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:59.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:58 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:58.979+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:19:59.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:59.196+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:19:59.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:59.196+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:19:59.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:59.196+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:19:59.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:19:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:19:59.196+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:19:59.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:58.997+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:19:59.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:58.997+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:19:59.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:58.997+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:19:59.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:58 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:58.997+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:19:59.665 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:59 smithi115 podman[73388]: 2024-03-26 18:19:59.401383489 +0000 UTC m=+6.073070099 container died e1a313ce9524d49a7893c94916b246173f4c10c37c4685989513f7591109d073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:19:59.949 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:59 smithi115 podman[73388]: 2024-03-26 18:19:59.774651076 +0000 UTC m=+6.446337677 container cleanup e1a313ce9524d49a7893c94916b246173f4c10c37c4685989513f7591109d073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:19:59.950 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:19:59 smithi115 bash[73388]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7 2024-03-26T18:19:59.950 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:59.975+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:19:59.950 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:59.975+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:19:59.950 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:59.975+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:19:59.951 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:59.975+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:19:59.951 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:19:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:19:59.975+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:00.229 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:20:00 smithi115 podman[73405]: 2024-03-26 18:20:00.067483021 +0000 UTC m=+0.669175804 container remove e1a313ce9524d49a7893c94916b246173f4c10c37c4685989513f7591109d073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-26T18:20:00.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:59 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:59.944+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:00.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:59 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:59.944+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:00.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:59 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:59.944+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:00.245 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:59 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:59.944+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:00.245 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:19:59 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:19:59.944+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:00.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:59.975+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:00.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:59.975+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:00.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:59.975+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:00.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:19:59 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:19:59.975+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:00.349 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:00.177+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:20:00.349 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:00.177+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:20:00.349 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:00.177+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:20:00.349 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:00.177+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:20:00.594 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:20:00 smithi115 podman[73499]: 2024-03-26 18:20:00.360169301 +0000 UTC m=+0.105097058 container create 6ffd8ebc4bbe6c314ddcaafd4b1f4dc6d35cd8cf4c0bb1307fd7278dde3711c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-26T18:20:00.594 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:20:00 smithi115 podman[73499]: 2024-03-26 18:20:00.272953791 +0000 UTC m=+0.017881528 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:20:00.594 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:20:00 smithi115 podman[73499]: 2024-03-26 18:20:00.620858139 +0000 UTC m=+0.365785870 container init 6ffd8ebc4bbe6c314ddcaafd4b1f4dc6d35cd8cf4c0bb1307fd7278dde3711c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-26T18:20:00.891 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:20:00 smithi115 podman[73499]: 2024-03-26 18:20:00.624227331 +0000 UTC m=+0.369155068 container start 6ffd8ebc4bbe6c314ddcaafd4b1f4dc6d35cd8cf4c0bb1307fd7278dde3711c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-26T18:20:00.891 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:20:00 smithi115 podman[73499]: 2024-03-26 18:20:00.729204129 +0000 UTC m=+0.474131877 container attach 6ffd8ebc4bbe6c314ddcaafd4b1f4dc6d35cd8cf4c0bb1307fd7278dde3711c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T18:20:01.185 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:00.952+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:01.185 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:00.952+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:01.185 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:00.952+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:01.185 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:00 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:00.952+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:01.197 INFO:journalctl@ceph.osd.7.smithi115.stdout:Mar 26 18:20:00 smithi115 podman[73499]: 2024-03-26 18:20:00.917853265 +0000 UTC m=+0.662781021 container died 6ffd8ebc4bbe6c314ddcaafd4b1f4dc6d35cd8cf4c0bb1307fd7278dde3711c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-26T18:20:01.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:00.957+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:01.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:00.957+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:01.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:00.957+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:01.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:00.957+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:01.197 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:00 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:00.957+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:01.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:00.983+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:01.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:00.983+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:01.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:00.983+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:01.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:00.983+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:01.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:00 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:00.983+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:01.512 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.7.service' 2024-03-26T18:20:01.595 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:01.596 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-26T18:20:01.596 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-26T18:20:01.596 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.8 2024-03-26T18:20:01.597 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:01.211+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:20:01.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:01.211+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:20:01.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:01.211+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:20:01.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:01.211+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:20:01.919 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:01 smithi171 systemd[1]: Stopping Ceph osd.8 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:02.184 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:01.944+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:02.185 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:01.944+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:02.185 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:01.944+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:02.185 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:01.944+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:02.185 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:01.954+0000 7f35b4cb7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:02.185 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:01.954+0000 7f35b4cb7700 -1 osd.8 119 *** Got signal Terminated *** 2024-03-26T18:20:02.185 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:01 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:01.954+0000 7f35b4cb7700 -1 osd.8 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:20:02.185 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:02.210+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:20:02.185 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:02.210+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:20:02.186 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:02.210+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:20:02.186 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:02.210+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:20:02.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:01.914+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:02.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:01.914+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:02.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:01.914+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:02.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:01.914+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:02.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:01 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:01.914+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:02.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:01 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:01.946+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:02.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:01 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:01.946+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:02.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:01 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:01.946+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:02.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:01 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:01.946+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:02.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:01 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:01.946+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:02.978 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:02.877+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:02.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:02.877+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:02.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:02.877+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:02.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:02.877+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:02.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:02 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:02.877+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:03.195 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:02.932+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:03.196 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:02.932+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:03.196 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:02.932+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:03.196 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:02.932+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:03.196 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:02 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:02.932+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:03.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:02 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:02.925+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:03.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:02 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:02.925+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:03.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:02 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:02.925+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:03.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:02 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:02.925+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:03.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:02 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:02.925+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:03.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:03.221+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:20:03.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:03.221+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:20:03.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:03.221+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:20:03.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:03.221+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:20:04.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:03.834+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:04.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:03.834+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:04.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:03.834+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:04.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:03.834+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:04.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:03.834+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:04.241 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:03.972+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:04.241 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:03.972+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:04.241 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:03.972+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:04.241 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:03.972+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:04.241 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:03 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:03.972+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:04.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:03 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:03.887+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:04.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:03 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:03.887+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:04.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:03 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:03.887+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:04.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:03 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:03.887+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:04.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:03 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:03.887+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:04.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:04.267+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:20:04.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:04.267+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:20:04.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:04.267+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:20:04.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:04.267+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:20:04.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:04.267+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.573828+0000 front 2024-03-26T18:19:37.573865+0000 (oldest deadline 2024-03-26T18:20:03.473892+0000) 2024-03-26T18:20:05.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:04 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:04.853+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:05.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:04 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:04.853+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:05.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:04 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:04.853+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:05.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:04 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:04.853+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:05.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:04 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:04.853+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:05.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:04 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:04.855+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:05.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:04 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:04.855+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:05.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:04 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:04.855+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:05.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:04 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:04.855+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:05.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:04 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:04.855+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:05.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:05 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:05.223+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:20:05.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:05 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:05.223+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:20:05.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:05 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:05.223+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:20:05.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:05 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:05.223+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:20:05.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:05 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:05.223+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.573828+0000 front 2024-03-26T18:19:37.573865+0000 (oldest deadline 2024-03-26T18:20:03.473892+0000) 2024-03-26T18:20:05.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:04.983+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:05.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:04.983+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:05.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:04.983+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:05.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:04.983+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:05.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:04 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:04.983+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:06.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:05 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:05.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:06.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:05 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:05.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:06.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:05 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:05.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:06.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:05 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:05.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:06.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:05 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:05.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:06.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:05 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:05.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:06.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:05 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:05.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:06.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:05 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:05.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:06.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:05 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:05.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:06.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:05 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:05.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:06.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:06.182+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:04.968520+0000 front 2024-03-26T18:19:04.968539+0000 (oldest deadline 2024-03-26T18:19:29.569314+0000) 2024-03-26T18:20:06.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:06.182+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:14.170871+0000 front 2024-03-26T18:19:14.170997+0000 (oldest deadline 2024-03-26T18:19:40.071039+0000) 2024-03-26T18:20:06.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:06.182+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:20.071720+0000 front 2024-03-26T18:19:20.071732+0000 (oldest deadline 2024-03-26T18:19:45.971443+0000) 2024-03-26T18:20:06.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:06.182+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:31.872418+0000 front 2024-03-26T18:19:31.872367+0000 (oldest deadline 2024-03-26T18:19:55.272729+0000) 2024-03-26T18:20:06.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8[54539]: 2024-03-26T18:20:06.182+0000 7f35b12ab700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.573828+0000 front 2024-03-26T18:19:37.573865+0000 (oldest deadline 2024-03-26T18:20:03.473892+0000) 2024-03-26T18:20:06.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:06.006+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:06.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:06.006+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:06.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:06.006+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:06.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:06.006+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:06.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:06 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:06.006+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:07.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:06 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:06.885+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:07.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:06 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:06.885+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:07.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:06 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:06.885+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:07.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:06 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:06.885+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:07.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:06 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:06.885+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:07.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:06 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:06.885+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:07.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:06.845+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:07.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:06.845+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:07.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:06.845+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:07.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:06.845+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:07.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:06 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:06.845+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:07.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:07.024+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:07.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:07.024+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:07.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:07.024+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:07.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:07.024+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:07.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:07.024+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:07.966 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:07 smithi171 podman[74199]: 2024-03-26 18:20:07.72886168 +0000 UTC m=+5.898688643 container died 006fdd7ab605ed228f382a16bd84235af2ca7c910fcc2d95883c86c42092602f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-26T18:20:08.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:07 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:07.846+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:08.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:07 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:07.846+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:08.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:07 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:07.846+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:08.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:07 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:07.846+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:08.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:07 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:07.846+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:08.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:07 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:07.846+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:08.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:07.833+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:08.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:07.833+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:08.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:07.833+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:08.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:07.833+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:08.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:07.833+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:08.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:07 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:07.833+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.696195+0000 front 2024-03-26T18:19:37.696306+0000 (oldest deadline 2024-03-26T18:20:07.096563+0000) 2024-03-26T18:20:08.347 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:08 smithi171 podman[74199]: 2024-03-26 18:20:08.12173395 +0000 UTC m=+6.291560905 container cleanup 006fdd7ab605ed228f382a16bd84235af2ca7c910fcc2d95883c86c42092602f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T18:20:08.348 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:08 smithi171 bash[74199]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8 2024-03-26T18:20:08.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:07.992+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:08.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:07.992+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:08.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:07.992+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:08.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:07.992+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:08.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:07.992+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:08.699 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:08 smithi171 podman[74214]: 2024-03-26 18:20:08.3901605 +0000 UTC m=+0.658804804 container remove 006fdd7ab605ed228f382a16bd84235af2ca7c910fcc2d95883c86c42092602f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T18:20:08.954 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:08 smithi171 podman[74305]: 2024-03-26 18:20:08.62900816 +0000 UTC m=+0.017841113 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:20:08.954 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:08 smithi171 podman[74305]: 2024-03-26 18:20:08.741375351 +0000 UTC m=+0.130208334 container create 23a6abf8b9046417b37b6c794ba65b0365ad69ee3f4e31e2e8244423776aa4b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-26T18:20:09.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:08 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:08.879+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:09.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:08 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:08.879+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:09.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:08 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:08.879+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:09.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:08 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:08.879+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:09.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:08 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:08.879+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:09.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:08 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:08.879+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:09.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:08 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:08.794+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:09.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:08 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:08.794+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:09.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:08 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:08.794+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:09.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:08 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:08.794+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:09.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:08 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:08.794+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:09.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:08 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:08.794+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.696195+0000 front 2024-03-26T18:19:37.696306+0000 (oldest deadline 2024-03-26T18:20:07.096563+0000) 2024-03-26T18:20:09.271 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:09 smithi171 podman[74305]: 2024-03-26 18:20:09.003336167 +0000 UTC m=+0.392169106 container init 23a6abf8b9046417b37b6c794ba65b0365ad69ee3f4e31e2e8244423776aa4b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T18:20:09.271 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:09 smithi171 podman[74305]: 2024-03-26 18:20:09.006444584 +0000 UTC m=+0.395277529 container start 23a6abf8b9046417b37b6c794ba65b0365ad69ee3f4e31e2e8244423776aa4b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0) 2024-03-26T18:20:09.271 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:09 smithi171 podman[74305]: 2024-03-26 18:20:09.094610387 +0000 UTC m=+0.483443335 container attach 23a6abf8b9046417b37b6c794ba65b0365ad69ee3f4e31e2e8244423776aa4b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-26T18:20:09.271 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:08 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:08.998+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:09.271 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:08 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:08.998+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:09.271 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:08 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:08.998+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:09.271 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:08 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:08.998+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:09.271 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:08 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:08.998+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:09.555 INFO:journalctl@ceph.osd.8.smithi171.stdout:Mar 26 18:20:09 smithi171 podman[74305]: 2024-03-26 18:20:09.297251767 +0000 UTC m=+0.686084712 container died 23a6abf8b9046417b37b6c794ba65b0365ad69ee3f4e31e2e8244423776aa4b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-26T18:20:09.879 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.8.service' 2024-03-26T18:20:09.919 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:09.920 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-26T18:20:09.920 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-26T18:20:09.920 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.9 2024-03-26T18:20:10.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:09 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:09.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:10.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:09 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:09.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:10.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:09 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:09.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:10.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:09 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:09.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:10.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:09 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:09.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:10.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:09 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:09.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:10.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:09 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:09.814+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:10.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:09 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:09.814+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:10.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:09 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:09.814+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:10.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:09 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:09.814+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:10.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:09 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:09.814+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:10.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:09 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:09.814+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.696195+0000 front 2024-03-26T18:19:37.696306+0000 (oldest deadline 2024-03-26T18:20:07.096563+0000) 2024-03-26T18:20:10.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:10 smithi049 systemd[1]: Stopping Ceph osd.9 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:10.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:09 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:09.954+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:10.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:09 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:09.954+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:10.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:09 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:09.954+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:10.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:09 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:09.954+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:10.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:09 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:09.954+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:10.744 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:10.274+0000 7f112a129700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:10.744 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:10.274+0000 7f112a129700 -1 osd.9 119 *** Got signal Terminated *** 2024-03-26T18:20:10.744 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:10.274+0000 7f112a129700 -1 osd.9 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:20:11.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:10 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:10.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:11.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:10 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:10.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:11.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:10 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:10.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:11.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:10 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:10.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:11.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:10 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:10.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:11.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:10 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:10.871+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:11.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:10.840+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:11.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:10.840+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:11.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:10.840+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:11.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:10.840+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:11.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:10.840+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:11.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:10 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:10.840+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.696195+0000 front 2024-03-26T18:19:37.696306+0000 (oldest deadline 2024-03-26T18:20:07.096563+0000) 2024-03-26T18:20:11.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:10 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:10.927+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:11.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:10 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:10.927+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:11.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:10 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:10.927+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:11.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:10 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:10.927+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:11.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:10 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:10.927+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:12.198 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:11 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:11.886+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:12.198 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:11 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:11.886+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:12.198 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:11 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:11.886+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:12.198 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:11 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:11.886+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:12.198 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:11 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:11.886+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:12.198 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:11 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:11.886+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:12.198 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:11 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:11.886+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:12.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:11 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:11.847+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:12.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:11 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:11.847+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:12.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:11 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:11.847+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:12.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:11 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:11.847+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:12.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:11 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:11.847+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:12.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:11 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:11.847+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.696195+0000 front 2024-03-26T18:19:37.696306+0000 (oldest deadline 2024-03-26T18:20:07.096563+0000) 2024-03-26T18:20:12.347 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:11 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:11.902+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:12.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:11 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:11.902+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:12.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:11 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:11.902+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:12.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:11 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:11.902+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:12.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:11 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:11.902+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:12.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:11 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:11.902+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:13.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:12 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:12.895+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:13.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:12 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:12.895+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:13.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:12 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:12.895+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:13.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:12 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:12.895+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:13.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:12 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:12.895+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:13.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:12 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:12.895+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:13.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:12 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:12.895+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:13.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:12.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:13.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:12.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:13.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:12.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:13.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:12.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:13.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:12.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:13.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:12 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:12.877+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.696195+0000 front 2024-03-26T18:19:37.696306+0000 (oldest deadline 2024-03-26T18:20:07.096563+0000) 2024-03-26T18:20:13.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:12 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:12.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:13.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:12 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:12.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:13.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:12 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:12.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:13.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:12 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:12.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:13.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:12 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:12.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:13.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:12 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:12.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:14.230 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:13 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:13.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:14.231 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:13 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:13.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:14.231 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:13 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:13.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:14.231 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:13 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:13.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:14.231 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:13 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:13.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:14.231 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:13 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:13.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:14.231 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:13 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:13.868+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:14.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:13 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:13.849+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:14.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:13 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:13.849+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:14.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:13 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:13.849+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:14.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:13 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:13.849+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:14.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:13 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:13.849+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:14.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:13 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:13.849+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.696195+0000 front 2024-03-26T18:19:37.696306+0000 (oldest deadline 2024-03-26T18:20:07.096563+0000) 2024-03-26T18:20:14.347 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:13 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:13.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:14.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:13 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:13.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:14.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:13 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:13.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:14.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:13 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:13.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:14.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:13 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:13.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:14.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:13 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:13.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:15.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:14 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:14.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:15.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:14 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:14.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:15.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:14 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:14.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:15.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:14 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:14.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:15.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:14 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:14.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:15.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:14 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:14.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:15.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:14 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:14.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:15.243 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:14 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:14.861+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:56.190087+0000 front 2024-03-26T18:18:56.190034+0000 (oldest deadline 2024-03-26T18:19:20.290187+0000) 2024-03-26T18:20:15.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:14 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:14.861+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.591687+0000 front 2024-03-26T18:19:05.591589+0000 (oldest deadline 2024-03-26T18:19:31.491331+0000) 2024-03-26T18:20:15.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:14 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:14.861+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:11.492055+0000 front 2024-03-26T18:19:11.491941+0000 (oldest deadline 2024-03-26T18:19:37.391728+0000) 2024-03-26T18:20:15.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:14 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:14.861+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.293698+0000 front 2024-03-26T18:19:24.293721+0000 (oldest deadline 2024-03-26T18:19:50.694613+0000) 2024-03-26T18:20:15.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:14 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:14.861+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.695412+0000 front 2024-03-26T18:19:30.695314+0000 (oldest deadline 2024-03-26T18:19:54.795221+0000) 2024-03-26T18:20:15.244 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:14 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9[67024]: 2024-03-26T18:20:14.861+0000 7f112671d700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:37.696195+0000 front 2024-03-26T18:19:37.696306+0000 (oldest deadline 2024-03-26T18:20:07.096563+0000) 2024-03-26T18:20:15.347 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:14 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:14.904+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:15.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:14 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:14.904+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:15.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:14 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:14.904+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:15.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:14 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:14.904+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:15.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:14 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:14.904+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:15.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:14 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:14.904+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:16.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:15 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:15.852+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:16.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:15 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:15.852+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:16.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:15 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:15.852+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:16.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:15 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:15.852+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:16.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:15 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:15.852+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:16.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:15 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:15.852+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:16.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:15 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:15.852+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:16.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:15 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:15.888+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:16.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:15 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:15.888+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:16.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:15 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:15.888+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:16.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:15 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:15.888+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:16.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:15 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:15.888+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:16.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:15 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:15.888+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:16.441 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:16 smithi049 podman[93919]: 2024-03-26 18:20:16.197419396 +0000 UTC m=+6.044148248 container died 5851a079df2f048f131f7e8d4f8280f297254a2da24f8ec4a516844febbce0c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-26T18:20:16.743 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:16 smithi049 podman[93919]: 2024-03-26 18:20:16.550407503 +0000 UTC m=+6.397136360 container cleanup 5851a079df2f048f131f7e8d4f8280f297254a2da24f8ec4a516844febbce0c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-26T18:20:16.743 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:16 smithi049 bash[93919]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9 2024-03-26T18:20:17.161 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:16 smithi049 podman[93937]: 2024-03-26 18:20:16.868390642 +0000 UTC m=+0.669498210 container remove 5851a079df2f048f131f7e8d4f8280f297254a2da24f8ec4a516844febbce0c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-26T18:20:17.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:16 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:16.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:17.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:16 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:16.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:17.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:16 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:16.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:17.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:16 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:16.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:17.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:16 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:16.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:17.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:16 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:16.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:17.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:16 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:16.836+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:17.347 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:16 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:16.917+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:17.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:16 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:16.917+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:17.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:16 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:16.917+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:17.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:16 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:16.917+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:17.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:16 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:16.917+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:17.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:16 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:16.917+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:17.486 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:17 smithi049 podman[94028]: 2024-03-26 18:20:17.091006485 +0000 UTC m=+0.018256146 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:20:17.487 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:17 smithi049 podman[94028]: 2024-03-26 18:20:17.202654966 +0000 UTC m=+0.129904660 container create d7693f79aa6267c6c8d43e7fd539b23a93868df8259c1935c3f8a18291d66699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-26T18:20:17.487 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:17 smithi049 podman[94028]: 2024-03-26 18:20:17.430392745 +0000 UTC m=+0.357642398 container init d7693f79aa6267c6c8d43e7fd539b23a93868df8259c1935c3f8a18291d66699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-26T18:20:17.487 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:17 smithi049 podman[94028]: 2024-03-26 18:20:17.433720487 +0000 UTC m=+0.360970139 container start d7693f79aa6267c6c8d43e7fd539b23a93868df8259c1935c3f8a18291d66699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:20:17.487 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:17 smithi049 podman[94028]: 2024-03-26 18:20:17.513432688 +0000 UTC m=+0.440682340 container attach d7693f79aa6267c6c8d43e7fd539b23a93868df8259c1935c3f8a18291d66699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-26T18:20:17.743 INFO:journalctl@ceph.osd.9.smithi049.stdout:Mar 26 18:20:17 smithi049 podman[94028]: 2024-03-26 18:20:17.738427116 +0000 UTC m=+0.665676768 container died d7693f79aa6267c6c8d43e7fd539b23a93868df8259c1935c3f8a18291d66699 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-26T18:20:18.130 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:17 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:17.856+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:18.130 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:17 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:17.856+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:18.130 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:17 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:17.856+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:18.130 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:17 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:17.856+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:18.130 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:17 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:17.856+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:18.130 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:17 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:17.856+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:18.131 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:17 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:17.856+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:18.286 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.9.service' 2024-03-26T18:20:18.336 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:18.337 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-26T18:20:18.337 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-26T18:20:18.337 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.10 2024-03-26T18:20:18.347 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:17 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:17.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:18.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:17 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:17.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:18.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:17 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:17.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:18.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:17 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:17.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:18.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:17 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:17.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:18.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:17 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:17.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:18.653 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 systemd[1]: Stopping Ceph osd.10 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:18.978 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:18.708+0000 7f3149e4a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:18.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:18.708+0000 7f3149e4a700 -1 osd.10 119 *** Got signal Terminated *** 2024-03-26T18:20:18.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:18.708+0000 7f3149e4a700 -1 osd.10 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:20:18.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:18.859+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:18.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:18.859+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:18.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:18.859+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:18.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:18.859+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:18.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:18.859+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:18.980 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:18.859+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:18.980 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:18 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:18.859+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:19.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:18 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:18.931+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:19.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:18 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:18.931+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:19.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:18 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:18.931+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:19.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:18 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:18.931+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:19.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:18 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:18.931+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:19.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:18 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:18.931+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:19.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:19 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:19.844+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:19.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:19 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:19.844+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:19.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:19 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:19.844+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:19.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:19 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:19.844+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:19.979 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:19 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:19.844+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:19.980 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:19 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:19.844+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:19.980 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:19 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:19.844+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:20.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:19 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:19.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:20.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:19 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:19.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:20.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:19 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:19.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:20.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:19 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:19.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:20.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:19 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:19.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:20.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:19 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:19.916+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:21.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:20 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:20.816+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:21.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:20 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:20.816+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:21.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:20 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:20.816+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:21.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:20 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:20.817+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:21.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:20 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:20.817+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:21.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:20 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:20.817+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:21.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:20 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:20.817+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:21.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:20 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:20.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:21.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:20 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:20.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:21.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:20 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:20.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:21.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:20 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:20.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:21.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:20 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:20.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:21.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:20 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:20.911+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:22.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:21 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:21.848+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:22.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:21 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:21.848+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:22.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:21 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:21.848+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:22.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:21 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:21.848+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:22.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:21 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:21.848+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:22.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:21 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:21.848+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:22.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:21 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:21.848+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:22.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:21 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:21.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:22.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:21 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:21.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:22.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:21 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:21.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:22.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:21 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:21.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:22.348 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:21 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:21.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:22.349 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:21 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:21.898+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:23.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:22 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:22.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:23.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:22 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:22.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:23.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:22 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:22.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:23.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:22 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:22.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:23.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:22 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:22.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:23.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:22 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:22.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:23.228 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:22 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:22.829+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:55.539658+0000 front 2024-03-26T18:18:55.539749+0000 (oldest deadline 2024-03-26T18:19:17.239580+0000) 2024-03-26T18:20:23.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:22 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:22.829+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:05.341684+0000 front 2024-03-26T18:19:05.341711+0000 (oldest deadline 2024-03-26T18:19:27.641878+0000) 2024-03-26T18:20:23.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:22 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:22.829+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:12.942857+0000 front 2024-03-26T18:19:12.943049+0000 (oldest deadline 2024-03-26T18:19:38.842756+0000) 2024-03-26T18:20:23.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:22 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:22.829+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2024-03-26T18:19:24.143999+0000 front 2024-03-26T18:19:24.143821+0000 (oldest deadline 2024-03-26T18:19:45.843833+0000) 2024-03-26T18:20:23.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:22 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:22.829+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:30.544975+0000 front 2024-03-26T18:19:30.544938+0000 (oldest deadline 2024-03-26T18:19:56.345539+0000) 2024-03-26T18:20:23.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:22 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:22.829+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:36.346122+0000 front 2024-03-26T18:19:36.346243+0000 (oldest deadline 2024-03-26T18:20:06.346371+0000) 2024-03-26T18:20:23.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:22 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10[60769]: 2024-03-26T18:20:22.829+0000 7f3142436700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:46.347258+0000 front 2024-03-26T18:19:46.347181+0000 (oldest deadline 2024-03-26T18:20:10.947253+0000) 2024-03-26T18:20:24.097 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:23 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:23.828+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:24.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:23 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:23.828+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:24.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:23 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:23.828+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:24.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:23 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:23.828+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:24.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:23 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:23.828+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:24.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:23 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:23.828+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:24.728 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:24 smithi115 podman[73617]: 2024-03-26 18:20:24.454478841 +0000 UTC m=+5.853331294 container died 913fad27d5a1ff5174e252b9f82836e94a2938827189ff8b7898587cf7c967f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-26T18:20:25.099 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:24.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:25.099 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:24.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:25.099 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:24.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:25.099 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:24.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:25.099 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:24.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:25.099 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:24.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:25.099 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:24 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:24.874+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-03-26T18:19:56.527224+0000 front 2024-03-26T18:19:56.527277+0000 (oldest deadline 2024-03-26T18:20:24.127431+0000) 2024-03-26T18:20:25.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:24 smithi115 podman[73617]: 2024-03-26 18:20:24.757490899 +0000 UTC m=+6.156343329 container cleanup 913fad27d5a1ff5174e252b9f82836e94a2938827189ff8b7898587cf7c967f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T18:20:25.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:24 smithi115 bash[73617]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10 2024-03-26T18:20:25.229 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:24 smithi115 podman[73649]: 2024-03-26 18:20:24.983844806 +0000 UTC m=+0.525944913 container remove 913fad27d5a1ff5174e252b9f82836e94a2938827189ff8b7898587cf7c967f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-26T18:20:25.510 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:25 smithi115 podman[73741]: 2024-03-26 18:20:25.267935515 +0000 UTC m=+0.106490438 container create 5920e8d0d632e36fc54191acb48fc5fcfdae964f2602068f484d565a95d262ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T18:20:25.510 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:25 smithi115 podman[73741]: 2024-03-26 18:20:25.17892336 +0000 UTC m=+0.017478282 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:20:25.510 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:25 smithi115 podman[73741]: 2024-03-26 18:20:25.461971315 +0000 UTC m=+0.300526228 container init 5920e8d0d632e36fc54191acb48fc5fcfdae964f2602068f484d565a95d262ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux ) 2024-03-26T18:20:25.510 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:25 smithi115 podman[73741]: 2024-03-26 18:20:25.465461931 +0000 UTC m=+0.304016829 container start 5920e8d0d632e36fc54191acb48fc5fcfdae964f2602068f484d565a95d262ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True) 2024-03-26T18:20:25.510 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:25 smithi115 podman[73741]: 2024-03-26 18:20:25.536534291 +0000 UTC m=+0.375089215 container attach 5920e8d0d632e36fc54191acb48fc5fcfdae964f2602068f484d565a95d262ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:20:25.978 INFO:journalctl@ceph.osd.10.smithi115.stdout:Mar 26 18:20:25 smithi115 podman[73741]: 2024-03-26 18:20:25.76642112 +0000 UTC m=+0.604976031 container died 5920e8d0d632e36fc54191acb48fc5fcfdae964f2602068f484d565a95d262ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-26T18:20:26.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:25 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:25.832+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:26.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:25 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:25.832+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:26.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:25 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:25.832+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:26.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:25 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:25.832+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:26.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:25 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:25.832+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:26.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:25 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:25.832+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:26.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:25 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:25.832+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-03-26T18:19:56.527224+0000 front 2024-03-26T18:19:56.527277+0000 (oldest deadline 2024-03-26T18:20:24.127431+0000) 2024-03-26T18:20:26.288 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.10.service' 2024-03-26T18:20:26.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:26.330 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-26T18:20:26.330 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-26T18:20:26.330 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.11 2024-03-26T18:20:26.659 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 systemd[1]: Stopping Ceph osd.11 for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:27.102 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:26.685+0000 7fc5457f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:27.102 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:26.685+0000 7fc5457f7700 -1 osd.11 119 *** Got signal Terminated *** 2024-03-26T18:20:27.102 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:26.685+0000 7fc5457f7700 -1 osd.11 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:20:27.102 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:26.863+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:27.102 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:26.863+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:27.102 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:26.863+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:27.102 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:26.863+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:27.102 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:26.863+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:27.103 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:26.863+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:27.103 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:26 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:26.863+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-03-26T18:19:56.527224+0000 front 2024-03-26T18:19:56.527277+0000 (oldest deadline 2024-03-26T18:20:24.127431+0000) 2024-03-26T18:20:28.097 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:27 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:27.860+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:28.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:27 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:27.860+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:28.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:27 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:27.860+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:28.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:27 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:27.860+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:28.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:27 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:27.860+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:28.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:27 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:27.860+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:28.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:27 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:27.860+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-03-26T18:19:56.527224+0000 front 2024-03-26T18:19:56.527277+0000 (oldest deadline 2024-03-26T18:20:24.127431+0000) 2024-03-26T18:20:29.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:28 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:28.846+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:29.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:28 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:28.846+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:29.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:28 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:28.846+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:29.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:28 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:28.846+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:29.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:28 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:28.846+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:29.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:28 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:28.846+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:29.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:28 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:28.846+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-03-26T18:19:56.527224+0000 front 2024-03-26T18:19:56.527277+0000 (oldest deadline 2024-03-26T18:20:24.127431+0000) 2024-03-26T18:20:30.097 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:29 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:29.807+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:30.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:29 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:29.807+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:30.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:29 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:29.807+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:30.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:29 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:29.807+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:30.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:29 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:29.807+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:30.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:29 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:29.807+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:30.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:29 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:29.807+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-03-26T18:19:56.527224+0000 front 2024-03-26T18:19:56.527277+0000 (oldest deadline 2024-03-26T18:20:24.127431+0000) 2024-03-26T18:20:31.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:30.801+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2024-03-26T18:18:54.113463+0000 front 2024-03-26T18:18:54.113381+0000 (oldest deadline 2024-03-26T18:19:18.213480+0000) 2024-03-26T18:20:31.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:30.801+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2024-03-26T18:19:06.315918+0000 front 2024-03-26T18:19:06.315934+0000 (oldest deadline 2024-03-26T18:19:32.216043+0000) 2024-03-26T18:20:31.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:30.801+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-03-26T18:19:13.917408+0000 front 2024-03-26T18:19:13.917355+0000 (oldest deadline 2024-03-26T18:19:36.718138+0000) 2024-03-26T18:20:31.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:30.801+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2024-03-26T18:19:29.222612+0000 front 2024-03-26T18:19:29.222871+0000 (oldest deadline 2024-03-26T18:19:53.922549+0000) 2024-03-26T18:20:31.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:30.801+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-03-26T18:19:38.023448+0000 front 2024-03-26T18:19:38.023481+0000 (oldest deadline 2024-03-26T18:20:02.723503+0000) 2024-03-26T18:20:31.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:30.801+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2024-03-26T18:19:48.926133+0000 front 2024-03-26T18:19:48.926110+0000 (oldest deadline 2024-03-26T18:20:11.826178+0000) 2024-03-26T18:20:31.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:30 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11[61317]: 2024-03-26T18:20:30.801+0000 7fc541deb700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.115:6820 osd.7 since back 2024-03-26T18:19:56.527224+0000 front 2024-03-26T18:19:56.527277+0000 (oldest deadline 2024-03-26T18:20:24.127431+0000) 2024-03-26T18:20:32.626 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:32 smithi171 podman[74414]: 2024-03-26 18:20:32.390543964 +0000 UTC m=+5.837663800 container died 8a2aa97ad59b5a759042529c0808c5bf095df947cf53a770de7ef6579f43c13c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T18:20:33.010 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:32 smithi171 podman[74414]: 2024-03-26 18:20:32.744194963 +0000 UTC m=+6.191314773 container cleanup 8a2aa97ad59b5a759042529c0808c5bf095df947cf53a770de7ef6579f43c13c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-26T18:20:33.010 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:32 smithi171 bash[74414]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11 2024-03-26T18:20:33.324 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:33 smithi171 podman[74431]: 2024-03-26 18:20:33.037352786 +0000 UTC m=+0.648225394 container remove 8a2aa97ad59b5a759042529c0808c5bf095df947cf53a770de7ef6579f43c13c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:20:33.598 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:33 smithi171 podman[74521]: 2024-03-26 18:20:33.254847528 +0000 UTC m=+0.017971692 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:20:33.598 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:33 smithi171 podman[74521]: 2024-03-26 18:20:33.355225746 +0000 UTC m=+0.118349940 container create 16649974ccb05877345266b3289b752c60b2095aeeef7656519f42ef3c0b781d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-03-26T18:20:33.598 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:33 smithi171 podman[74521]: 2024-03-26 18:20:33.57507579 +0000 UTC m=+0.338199945 container init 16649974ccb05877345266b3289b752c60b2095aeeef7656519f42ef3c0b781d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-26T18:20:33.598 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:33 smithi171 podman[74521]: 2024-03-26 18:20:33.578538232 +0000 UTC m=+0.341662385 container start 16649974ccb05877345266b3289b752c60b2095aeeef7656519f42ef3c0b781d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:20:34.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:33 smithi171 podman[74521]: 2024-03-26 18:20:33.658587908 +0000 UTC m=+0.421712084 container attach 16649974ccb05877345266b3289b752c60b2095aeeef7656519f42ef3c0b781d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:20:34.098 INFO:journalctl@ceph.osd.11.smithi171.stdout:Mar 26 18:20:33 smithi171 podman[74521]: 2024-03-26 18:20:33.87028458 +0000 UTC m=+0.633408735 container died 16649974ccb05877345266b3289b752c60b2095aeeef7656519f42ef3c0b781d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T18:20:34.428 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@osd.11.service' 2024-03-26T18:20:34.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:34.469 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-26T18:20:34.469 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-26T18:20:34.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.a 2024-03-26T18:20:34.767 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:20:34 smithi049 systemd[1]: Stopping Ceph mds.a for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:35.244 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:20:34 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a[73837]: 2024-03-26T18:20:34.793+0000 7f65ec0dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:35.244 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:20:34 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a[73837]: 2024-03-26T18:20:34.793+0000 7f65ec0dc700 -1 mds.a *** got signal Terminated *** 2024-03-26T18:20:38.542 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:20:38 smithi049 podman[94145]: 2024-03-26 18:20:38.313944685 +0000 UTC m=+3.614928871 container died e76a9c150d1beacb54cb29078a5aa93cbf184c52ae32a2bdeccdf4f3316382e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319) 2024-03-26T18:20:38.817 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:20:38 smithi049 podman[94145]: 2024-03-26 18:20:38.659617063 +0000 UTC m=+3.960601246 container cleanup e76a9c150d1beacb54cb29078a5aa93cbf184c52ae32a2bdeccdf4f3316382e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:20:38.817 INFO:journalctl@ceph.mds.a.smithi049.stdout:Mar 26 18:20:38 smithi049 bash[94145]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-a 2024-03-26T18:20:39.033 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.a.service' 2024-03-26T18:20:39.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:39.073 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-26T18:20:39.074 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-26T18:20:39.074 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.d 2024-03-26T18:20:39.412 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 18:20:39 smithi049 systemd[1]: Stopping Ceph mds.d for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:39.743 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 18:20:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-d[72878]: 2024-03-26T18:20:39.438+0000 7fa0d661b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:39.744 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 18:20:39 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-d[72878]: 2024-03-26T18:20:39.438+0000 7fa0d661b700 -1 mds.d *** got signal Terminated *** 2024-03-26T18:20:40.821 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 18:20:40 smithi049 podman[94265]: 2024-03-26 18:20:40.595107816 +0000 UTC m=+1.264317061 container died c008ea39c957a90eedad4b2af4567e73bcb948bf313ac3b60a4ee5d8bc5bc0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-03-26T18:20:41.079 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 18:20:40 smithi049 podman[94265]: 2024-03-26 18:20:40.913472503 +0000 UTC m=+1.582681758 container cleanup c008ea39c957a90eedad4b2af4567e73bcb948bf313ac3b60a4ee5d8bc5bc0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-26T18:20:41.079 INFO:journalctl@ceph.mds.d.smithi049.stdout:Mar 26 18:20:40 smithi049 bash[94265]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-d 2024-03-26T18:20:41.211 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.d.service' 2024-03-26T18:20:41.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:41.251 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-26T18:20:41.251 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-26T18:20:41.251 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.g 2024-03-26T18:20:41.743 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 18:20:41 smithi049 systemd[1]: Stopping Ceph mds.g for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:41.744 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 18:20:41 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g[73474]: 2024-03-26T18:20:41.592+0000 7f69534f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:41.744 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 18:20:41 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g[73474]: 2024-03-26T18:20:41.592+0000 7f69534f7700 -1 mds.g *** got signal Terminated *** 2024-03-26T18:20:45.858 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 18:20:45 smithi049 podman[94386]: 2024-03-26 18:20:45.606794528 +0000 UTC m=+4.105778649 container died 052ef4e36fe03b656a71f9e9c66636519da3d2022e05a655796d22d47c16e3b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T18:20:46.224 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 18:20:45 smithi049 podman[94386]: 2024-03-26 18:20:45.975763221 +0000 UTC m=+4.474747341 container cleanup 052ef4e36fe03b656a71f9e9c66636519da3d2022e05a655796d22d47c16e3b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.vendor=CentOS) 2024-03-26T18:20:46.225 INFO:journalctl@ceph.mds.g.smithi049.stdout:Mar 26 18:20:45 smithi049 bash[94386]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-g 2024-03-26T18:20:46.383 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.g.service' 2024-03-26T18:20:46.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:46.423 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-26T18:20:46.423 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-26T18:20:46.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.j 2024-03-26T18:20:46.743 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 18:20:46 smithi049 systemd[1]: Stopping Ceph mds.j for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:47.243 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 18:20:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-j[72530]: 2024-03-26T18:20:46.786+0000 7fc999374700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:47.243 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 18:20:46 smithi049 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-j[72530]: 2024-03-26T18:20:46.786+0000 7fc999374700 -1 mds.j *** got signal Terminated *** 2024-03-26T18:20:49.927 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 18:20:49 smithi049 podman[94510]: 2024-03-26 18:20:49.67966577 +0000 UTC m=+2.998653014 container died c7eceaac0a995e8be72b1d49b0317d265ef7bda1953df745afbac9f45b9be88a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-j, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-26T18:20:50.243 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 18:20:50 smithi049 podman[94510]: 2024-03-26 18:20:50.037516085 +0000 UTC m=+3.356503326 container cleanup c7eceaac0a995e8be72b1d49b0317d265ef7bda1953df745afbac9f45b9be88a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-26T18:20:50.244 INFO:journalctl@ceph.mds.j.smithi049.stdout:Mar 26 18:20:50 smithi049 bash[94510]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-j 2024-03-26T18:20:50.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.j.service' 2024-03-26T18:20:50.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:50.422 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-26T18:20:50.422 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-26T18:20:50.422 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.b 2024-03-26T18:20:50.716 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:20:50 smithi115 systemd[1]: Stopping Ceph mds.b for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:50.978 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:20:50 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-b[66478]: 2024-03-26T18:20:50.753+0000 7fee6cfc9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:50.978 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:20:50 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-b[66478]: 2024-03-26T18:20:50.753+0000 7fee6cfc9700 -1 mds.b *** got signal Terminated *** 2024-03-26T18:20:54.576 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:20:54 smithi115 podman[73844]: 2024-03-26 18:20:54.281876519 +0000 UTC m=+3.638498323 container died ab1506df1fcc73788632b4bb0b595fbb0d436349f45df20338e552db63580dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-b, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:20:54.577 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:20:54 smithi115 podman[73844]: 2024-03-26 18:20:54.603681287 +0000 UTC m=+3.960303055 container cleanup ab1506df1fcc73788632b4bb0b595fbb0d436349f45df20338e552db63580dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-26T18:20:54.978 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:20:54 smithi115 bash[73844]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-b 2024-03-26T18:20:55.095 INFO:journalctl@ceph.mds.b.smithi115.stdout:Mar 26 18:20:54 smithi115 podman[73870]: 2024-03-26 18:20:54.813290014 +0000 UTC m=+0.525235500 container remove ab1506df1fcc73788632b4bb0b595fbb0d436349f45df20338e552db63580dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-26T18:20:55.097 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.b.service' 2024-03-26T18:20:55.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:55.151 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-26T18:20:55.151 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-26T18:20:55.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.e 2024-03-26T18:20:55.481 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 18:20:55 smithi115 systemd[1]: Stopping Ceph mds.e for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:55.978 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 18:20:55 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-e[65848]: 2024-03-26T18:20:55.506+0000 7f6f4db9f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:55.978 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 18:20:55 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-e[65848]: 2024-03-26T18:20:55.506+0000 7f6f4db9f700 -1 mds.e *** got signal Terminated *** 2024-03-26T18:20:58.522 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 18:20:58 smithi115 podman[73976]: 2024-03-26 18:20:58.284307645 +0000 UTC m=+2.881758619 container died 82c9270a6458d440ceb72572cb94cfc82bbd2149ac69bfbdfa0670947c013b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:20:58.889 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 18:20:58 smithi115 podman[73976]: 2024-03-26 18:20:58.648799338 +0000 UTC m=+3.246250264 container cleanup 82c9270a6458d440ceb72572cb94cfc82bbd2149ac69bfbdfa0670947c013b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-26T18:20:58.890 INFO:journalctl@ceph.mds.e.smithi115.stdout:Mar 26 18:20:58 smithi115 bash[73976]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-e 2024-03-26T18:20:59.037 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.e.service' 2024-03-26T18:20:59.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:20:59.073 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-26T18:20:59.073 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-26T18:20:59.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.h 2024-03-26T18:20:59.478 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 18:20:59 smithi115 systemd[1]: Stopping Ceph mds.h for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:20:59.478 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 18:20:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h[66157]: 2024-03-26T18:20:59.417+0000 7f8c272d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:20:59.479 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 18:20:59 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h[66157]: 2024-03-26T18:20:59.417+0000 7f8c272d1700 -1 mds.h *** got signal Terminated *** 2024-03-26T18:21:02.518 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 18:21:02 smithi115 podman[74099]: 2024-03-26 18:21:02.285736876 +0000 UTC m=+2.970554564 container died 2420a64b3c8eeecc8826d7beb2cba6406646addf44f5ee05fdcbb8eff29dbc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:21:02.888 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 18:21:02 smithi115 podman[74099]: 2024-03-26 18:21:02.635138645 +0000 UTC m=+3.319956333 container cleanup 2420a64b3c8eeecc8826d7beb2cba6406646addf44f5ee05fdcbb8eff29dbc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-26T18:21:02.888 INFO:journalctl@ceph.mds.h.smithi115.stdout:Mar 26 18:21:02 smithi115 bash[74099]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-h 2024-03-26T18:21:02.981 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.h.service' 2024-03-26T18:21:03.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:21:03.017 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-26T18:21:03.017 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-26T18:21:03.017 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.k 2024-03-26T18:21:03.478 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 18:21:03 smithi115 systemd[1]: Stopping Ceph mds.k for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:21:03.478 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 18:21:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k[65527]: 2024-03-26T18:21:03.355+0000 7f44cbb30700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:21:03.478 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 18:21:03 smithi115 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k[65527]: 2024-03-26T18:21:03.355+0000 7f44cbb30700 -1 mds.k *** got signal Terminated *** 2024-03-26T18:21:06.562 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 18:21:06 smithi115 podman[74220]: 2024-03-26 18:21:06.28769722 +0000 UTC m=+3.029994956 container died d6dc72aac7202ced1c8f012cc9d1850ddc8386970966ca8507f7981f7e0bef12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:21:06.820 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 18:21:06 smithi115 podman[74220]: 2024-03-26 18:21:06.680333774 +0000 UTC m=+3.422631461 container cleanup d6dc72aac7202ced1c8f012cc9d1850ddc8386970966ca8507f7981f7e0bef12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:21:06.820 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 18:21:06 smithi115 bash[74220]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k 2024-03-26T18:21:07.073 INFO:journalctl@ceph.mds.k.smithi115.stdout:Mar 26 18:21:06 smithi115 podman[74234]: 2024-03-26 18:21:06.847710454 +0000 UTC m=+0.469842006 container remove d6dc72aac7202ced1c8f012cc9d1850ddc8386970966ca8507f7981f7e0bef12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-26T18:21:07.076 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.k.service' 2024-03-26T18:21:07.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:21:07.112 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-26T18:21:07.112 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-26T18:21:07.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.c 2024-03-26T18:21:07.552 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 18:21:07 smithi171 systemd[1]: Stopping Ceph mds.c for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:21:07.847 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 18:21:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c[66804]: 2024-03-26T18:21:07.713+0000 7fe6b2290700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:21:07.847 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 18:21:07 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c[66804]: 2024-03-26T18:21:07.713+0000 7fe6b2290700 -1 mds.c *** got signal Terminated *** 2024-03-26T18:21:10.554 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 18:21:10 smithi171 podman[74626]: 2024-03-26 18:21:10.29882044 +0000 UTC m=+2.963979013 container died 259c053f8c6facf7da4f0a8dd0b944c45cb1aa7d10f202420315d3f0787cd7cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-26T18:21:10.847 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 18:21:10 smithi171 podman[74626]: 2024-03-26 18:21:10.671975619 +0000 UTC m=+3.337134186 container cleanup 259c053f8c6facf7da4f0a8dd0b944c45cb1aa7d10f202420315d3f0787cd7cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T18:21:10.847 INFO:journalctl@ceph.mds.c.smithi171.stdout:Mar 26 18:21:10 smithi171 bash[74626]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-c 2024-03-26T18:21:11.228 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.c.service' 2024-03-26T18:21:11.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:21:11.267 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-26T18:21:11.267 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-26T18:21:11.267 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.f 2024-03-26T18:21:11.597 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 18:21:11 smithi171 systemd[1]: Stopping Ceph mds.f for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:21:12.097 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 18:21:11 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f[66498]: 2024-03-26T18:21:11.634+0000 7fddf790c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:21:12.097 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 18:21:11 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f[66498]: 2024-03-26T18:21:11.634+0000 7fddf790c700 -1 mds.f *** got signal Terminated *** 2024-03-26T18:21:14.543 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 18:21:14 smithi171 podman[74760]: 2024-03-26 18:21:14.305439353 +0000 UTC m=+2.794647815 container died 3e7821f4b5e0c942f59d5a09b3db2fc0b13b81303ce0e09c1ebf843891683835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-26T18:21:14.847 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 18:21:14 smithi171 podman[74760]: 2024-03-26 18:21:14.652379398 +0000 UTC m=+3.141587856 container cleanup 3e7821f4b5e0c942f59d5a09b3db2fc0b13b81303ce0e09c1ebf843891683835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux ) 2024-03-26T18:21:14.847 INFO:journalctl@ceph.mds.f.smithi171.stdout:Mar 26 18:21:14 smithi171 bash[74760]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-f 2024-03-26T18:21:15.003 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.f.service' 2024-03-26T18:21:15.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:21:15.039 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-26T18:21:15.040 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-26T18:21:15.040 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.i 2024-03-26T18:21:15.338 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:21:15 smithi171 systemd[1]: Stopping Ceph mds.i for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:21:15.597 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:21:15 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i[67107]: 2024-03-26T18:21:15.365+0000 7f6941e31700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:21:15.597 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:21:15 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i[67107]: 2024-03-26T18:21:15.365+0000 7f6941e31700 -1 mds.i *** got signal Terminated *** 2024-03-26T18:21:18.547 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:21:18 smithi171 podman[74881]: 2024-03-26 18:21:18.30808712 +0000 UTC m=+3.023928159 container died 2f2c017b3885e772e5694488d33a50383ab61fdb0508176c1f1a07045d5f08cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-26T18:21:18.847 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:21:18 smithi171 podman[74881]: 2024-03-26 18:21:18.649101055 +0000 UTC m=+3.364942091 container cleanup 2f2c017b3885e772e5694488d33a50383ab61fdb0508176c1f1a07045d5f08cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-26T18:21:18.847 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:21:18 smithi171 bash[74881]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i 2024-03-26T18:21:18.847 INFO:journalctl@ceph.mds.i.smithi171.stdout:Mar 26 18:21:18 smithi171 podman[74894]: 2024-03-26 18:21:18.817140997 +0000 UTC m=+0.512008667 container remove 2f2c017b3885e772e5694488d33a50383ab61fdb0508176c1f1a07045d5f08cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-i, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-26T18:21:19.037 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.i.service' 2024-03-26T18:21:19.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:21:19.072 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-26T18:21:19.073 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-26T18:21:19.073 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.l 2024-03-26T18:21:19.347 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:21:19 smithi171 systemd[1]: Stopping Ceph mds.l for 5a3632a4-eb99-11ee-95d0-87774f69a715... 2024-03-26T18:21:19.847 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:21:19 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l[66177]: 2024-03-26T18:21:19.427+0000 7f1389691700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:21:19.847 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:21:19 smithi171 ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l[66177]: 2024-03-26T18:21:19.427+0000 7f1389691700 -1 mds.l *** got signal Terminated *** 2024-03-26T18:21:22.576 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:21:22 smithi171 podman[75002]: 2024-03-26 18:21:22.299474552 +0000 UTC m=+2.980963020 container died b3e097598a091b0729a802929fabd95cf7d2858a9bf41d9f11150c6a7a37a560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:21:22.576 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:21:22 smithi171 podman[75002]: 2024-03-26 18:21:22.603733424 +0000 UTC m=+3.285221854 container cleanup b3e097598a091b0729a802929fabd95cf7d2858a9bf41d9f11150c6a7a37a560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:21:22.847 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:21:22 smithi171 bash[75002]: ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l 2024-03-26T18:21:22.847 INFO:journalctl@ceph.mds.l.smithi171.stdout:Mar 26 18:21:22 smithi171 podman[75015]: 2024-03-26 18:21:22.813305386 +0000 UTC m=+0.514598068 container remove b3e097598a091b0729a802929fabd95cf7d2858a9bf41d9f11150c6a7a37a560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-5a3632a4-eb99-11ee-95d0-87774f69a715-mds-l, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2) 2024-03-26T18:21:22.903 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5a3632a4-eb99-11ee-95d0-87774f69a715@mds.l.service' 2024-03-26T18:21:22.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:21:22.938 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-26T18:21:22.938 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T18:21:23.142 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T18:21:26.167 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T18:21:26.356 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T18:21:28.874 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T18:21:29.067 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T18:21:31.491 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:21:31.530 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:21:31.564 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:21:31.593 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-26T18:21:31.595 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762/remote/smithi049/crash 2024-03-26T18:21:31.596 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/crash -- . 2024-03-26T18:21:31.645 INFO:teuthology.orchestra.run.smithi049.stderr:tar: /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T18:21:31.645 INFO:teuthology.orchestra.run.smithi049.stderr:tar: Error is not recoverable: exiting now 2024-03-26T18:21:31.647 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762/remote/smithi115/crash 2024-03-26T18:21:31.648 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/crash -- . 2024-03-26T18:21:31.721 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T18:21:31.722 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2024-03-26T18:21:31.724 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762/remote/smithi171/crash 2024-03-26T18:21:31.724 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/crash -- . 2024-03-26T18:21:31.761 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T18:21:31.761 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2024-03-26T18:21:31.762 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-26T18:21:31.762 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v '\(PG_DEGRADED' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-26T18:21:31.841 INFO:teuthology.orchestra.run.smithi049.stdout:2024-03-26T18:01:47.476275+0000 mon.a (mon.0) 1147 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-26T18:21:31.842 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-26T18:21:31.842 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[SEC\]' /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v '\(PG_DEGRADED' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-26T18:21:31.882 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]' /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v '\(PG_DEGRADED' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-26T18:21:31.962 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[WRN\]' /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v '\(PG_DEGRADED' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-26T18:21:32.042 INFO:teuthology.orchestra.run.smithi049.stdout:2024-03-26T18:01:47.476275+0000 mon.a (mon.0) 1147 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-26T18:21:32.042 INFO:tasks.cephadm:Compressing logs... 2024-03-26T18:21:32.042 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T18:21:32.084 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T18:21:32.087 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T18:21:32.130 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.l.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.i.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.l.log: /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.11.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mgr.z.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.8.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.f.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-26T18:21:32.145 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.c.log 2024-03-26T18:21:32.146 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T18:21:32.146 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-26T18:21:32.147 INFO:teuthology.orchestra.run.smithi049.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-03-26T18:21:32.147 INFO:teuthology.orchestra.run.smithi049.stderr: -- /var/log/ceph/cephadm.log 2024-03-26T18:21:32.147 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T18:21:32.148 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T18:21:32.148 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.10.log 2024-03-26T18:21:32.148 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T18:21:32.148 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.b.log 2024-03-26T18:21:32.148 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-26T18:21:32.151 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-26T18:21:32.151 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.k.log 2024-03-26T18:21:32.151 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.h.log 2024-03-26T18:21:32.151 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-26T18:21:32.151 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.e.log 2024-03-26T18:21:32.151 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-26T18:21:32.151 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T18:21:32.151 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log 2024-03-26T18:21:32.152 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T18:21:32.152 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-26T18:21:32.152 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mon.c.log: 92.2% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mgr.z.log.gz 2024-03-26T18:21:32.153 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-26T18:21:32.153 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-26T18:21:32.153 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T18:21:32.153 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.g.log 2024-03-26T18:21:32.153 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.j.log 2024-03-26T18:21:32.154 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-26T18:21:32.154 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mon.b.log: 92.1% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-26T18:21:32.156 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-26T18:21:32.156 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-26T18:21:32.156 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T18:21:32.156 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log 2024-03-26T18:21:32.156 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T18:21:32.156 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.9.log 2024-03-26T18:21:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.d.log 2024-03-26T18:21:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.a.log 2024-03-26T18:21:32.157 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-26T18:21:32.209 INFO:teuthology.orchestra.run.smithi115.stderr: 93.9% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.k.log.gz 2024-03-26T18:21:32.211 INFO:teuthology.orchestra.run.smithi115.stderr: 84.0% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T18:21:32.221 INFO:teuthology.orchestra.run.smithi171.stderr: 83.9% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T18:21:32.239 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.log: 84.3% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T18:21:32.246 INFO:teuthology.orchestra.run.smithi049.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-26T18:21:32.272 INFO:teuthology.orchestra.run.smithi049.stderr: 90.3% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T18:21:32.281 INFO:teuthology.orchestra.run.smithi171.stderr: 93.5% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.l.log.gz 2024-03-26T18:21:32.303 INFO:teuthology.orchestra.run.smithi049.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T18:21:32.310 INFO:teuthology.orchestra.run.smithi049.stderr: 94.6% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.d.log.gz 2024-03-26T18:21:32.320 INFO:teuthology.orchestra.run.smithi171.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T18:21:32.325 INFO:teuthology.orchestra.run.smithi171.stderr: 86.8% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T18:21:32.350 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T18:21:32.419 INFO:teuthology.orchestra.run.smithi049.stderr: 94.6% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.j.log.gz 2024-03-26T18:21:32.420 INFO:teuthology.orchestra.run.smithi049.stderr: 86.8% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T18:21:32.433 INFO:teuthology.orchestra.run.smithi171.stderr: 90.4% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T18:21:32.461 INFO:teuthology.orchestra.run.smithi171.stderr: 90.9% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T18:21:32.539 INFO:teuthology.orchestra.run.smithi115.stderr: 91.0% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T18:21:32.699 INFO:teuthology.orchestra.run.smithi171.stderr: 94.7% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.c.log.gz 2024-03-26T18:21:32.760 INFO:teuthology.orchestra.run.smithi115.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T18:21:32.763 INFO:teuthology.orchestra.run.smithi115.stderr: 86.8% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T18:21:32.766 INFO:teuthology.orchestra.run.smithi115.stderr: 90.4% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T18:21:33.200 INFO:teuthology.orchestra.run.smithi049.stderr: 94.5% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.g.log.gz 2024-03-26T18:21:33.612 INFO:teuthology.orchestra.run.smithi115.stderr: 94.0% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.e.log.gz 2024-03-26T18:21:33.612 INFO:teuthology.orchestra.run.smithi049.stderr: 94.6% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.a.log.gz 2024-03-26T18:21:33.995 INFO:teuthology.orchestra.run.smithi115.stderr: 95.3% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.h.log.gz 2024-03-26T18:21:35.563 INFO:teuthology.orchestra.run.smithi171.stderr: 92.2% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-26T18:21:36.588 INFO:teuthology.orchestra.run.smithi171.stderr: 94.4% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.f.log.gz 2024-03-26T18:21:36.940 INFO:teuthology.orchestra.run.smithi049.stderr: 90.5% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-26T18:21:39.829 INFO:teuthology.orchestra.run.smithi115.stderr: 91.8% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-26T18:21:40.736 INFO:teuthology.orchestra.run.smithi115.stderr: 93.5% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.b.log.gz 2024-03-26T18:21:48.851 INFO:teuthology.orchestra.run.smithi049.stderr: 90.3% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-26T18:22:27.341 INFO:teuthology.orchestra.run.smithi171.stderr: 93.9% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-mds.i.log.gz 2024-03-26T18:28:51.557 INFO:teuthology.orchestra.run.smithi115.stderr: 90.5% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.10.log.gz 2024-03-26T18:30:30.603 INFO:teuthology.orchestra.run.smithi171.stderr: 90.4% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.8.log.gz 2024-03-26T18:30:35.424 INFO:teuthology.orchestra.run.smithi115.stderr: 90.4% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-26T18:33:03.074 INFO:teuthology.orchestra.run.smithi115.stderr: 90.4% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-26T18:33:27.997 INFO:teuthology.orchestra.run.smithi171.stderr: 90.5% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.11.log.gz 2024-03-26T18:33:45.053 INFO:teuthology.orchestra.run.smithi115.stderr: 90.7% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-26T18:33:45.055 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-03-26T18:33:45.056 INFO:teuthology.orchestra.run.smithi115.stderr:real 12m12.944s 2024-03-26T18:33:45.056 INFO:teuthology.orchestra.run.smithi115.stderr:user 6m45.728s 2024-03-26T18:33:45.056 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m15.886s 2024-03-26T18:34:26.901 INFO:teuthology.orchestra.run.smithi171.stderr: 90.7% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-26T18:34:57.446 INFO:teuthology.orchestra.run.smithi171.stderr: 90.4% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-26T18:34:57.538 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-03-26T18:34:57.538 INFO:teuthology.orchestra.run.smithi171.stderr:real 13m25.411s 2024-03-26T18:34:57.538 INFO:teuthology.orchestra.run.smithi171.stderr:user 6m53.759s 2024-03-26T18:34:57.539 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m16.472s 2024-03-26T18:37:07.873 INFO:teuthology.orchestra.run.smithi049.stderr: 90.5% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-26T18:38:53.819 INFO:teuthology.orchestra.run.smithi049.stderr: 90.4% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-26T18:40:31.066 INFO:teuthology.orchestra.run.smithi049.stderr: 90.4% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-26T18:41:27.540 INFO:teuthology.orchestra.run.smithi049.stderr: 90.3% -- replaced with /var/log/ceph/5a3632a4-eb99-11ee-95d0-87774f69a715/ceph-osd.9.log.gz 2024-03-26T18:41:27.571 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-03-26T18:41:27.572 INFO:teuthology.orchestra.run.smithi049.stderr:real 19m55.458s 2024-03-26T18:41:27.572 INFO:teuthology.orchestra.run.smithi049.stderr:user 8m49.127s 2024-03-26T18:41:27.572 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m21.062s 2024-03-26T18:41:27.572 INFO:tasks.cephadm:Archiving logs... 2024-03-26T18:41:27.573 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762/remote/smithi049/log 2024-03-26T18:41:27.574 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T18:42:59.501 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762/remote/smithi115/log 2024-03-26T18:42:59.502 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T18:44:14.097 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762/remote/smithi171/log 2024-03-26T18:44:14.098 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T18:45:29.915 INFO:tasks.cephadm:Removing cluster... 2024-03-26T18:45:29.916 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 --force 2024-03-26T18:45:30.490 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T18:45:31.432 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 --force 2024-03-26T18:45:31.690 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T18:45:32.441 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5a3632a4-eb99-11ee-95d0-87774f69a715 --force 2024-03-26T18:45:32.834 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 5a3632a4-eb99-11ee-95d0-87774f69a715 2024-03-26T18:45:33.592 INFO:tasks.cephadm:Removing cephadm ... 2024-03-26T18:45:33.593 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T18:45:33.632 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T18:45:33.651 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T18:45:33.667 INFO:tasks.cephadm:Teardown complete 2024-03-26T18:45:33.668 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-26T18:45:33.681 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-26T18:45:33.681 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T18:45:33.684 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T18:45:33.694 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T18:45:33.873 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-26T18:45:33.884 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-26T18:45:33.884 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T18:45:33.887 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T18:45:33.889 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-03-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-03-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-03-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T18:45:33.940 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-03-26T18:45:33.940 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 50 +766us[ +832us] +/- 72ms 2024-03-26T18:45:33.940 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv02.front.sepia.ceph.com 3 6 377 48 -528us[ -462us] +/- 67ms 2024-03-26T18:45:33.940 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -156us[ -89us] +/- 86ms 2024-03-26T18:45:33.940 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-03-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 51 +744us[ +779us] +/- 72ms 2024-03-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -529us[ -494us] +/- 67ms 2024-03-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -168us[ -168us] +/- 86ms 2024-03-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-03-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 +740us[ +804us] +/- 72ms 2024-03-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -540us[ -476us] +/- 67ms 2024-03-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -165us[ -165us] +/- 86ms 2024-03-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:45:34.020 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-26T18:45:34.033 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-26T18:45:34.034 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-26T18:45:34.045 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T18:45:34.218 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T18:45:34.326 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T18:45:34.437 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T18:45:34.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:45:34.475 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T18:45:34.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:45:34.508 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T18:45:34.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:45:34.538 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-26T18:45:34.550 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-26T18:45:34.561 INFO:teuthology.task.internal:Duration was 3712.572109 seconds 2024-03-26T18:45:34.561 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-26T18:45:34.572 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-26T18:45:34.572 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T18:45:34.575 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T18:45:34.577 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T18:45:34.644 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T18:45:34.650 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T18:45:34.682 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T18:45:35.169 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-26T18:45:35.169 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-03-26T18:45:35.169 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T18:45:35.204 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-03-26T18:45:35.204 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T18:45:35.236 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-03-26T18:45:35.237 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T18:45:35.274 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-26T18:45:35.274 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T18:45:35.276 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T18:45:35.279 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T18:45:35.353 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-26T18:45:35.354 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T18:45:35.356 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T18:45:35.362 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T18:45:35.776 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-26T18:45:35.789 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-26T18:45:35.789 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T18:45:35.821 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T18:45:35.854 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T18:45:35.884 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-26T18:45:35.895 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:45:35.898 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:45:35.900 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:45:35.937 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-03-26T18:45:35.938 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-03-26T18:45:35.969 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-03-26T18:45:36.000 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:45:36.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:45:36.029 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:45:36.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:45:36.045 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:45:36.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:45:36.059 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-26T18:45:36.073 INFO:teuthology.task.internal:Transferring archived files... 2024-03-26T18:45:36.076 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762/remote/smithi049 2024-03-26T18:45:36.076 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T18:45:36.126 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762/remote/smithi115 2024-03-26T18:45:36.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T18:45:36.164 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762/remote/smithi171 2024-03-26T18:45:36.165 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T18:45:36.204 INFO:teuthology.task.internal:Removing archive directory... 2024-03-26T18:45:36.205 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T18:45:36.207 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T18:45:36.210 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T18:45:36.253 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-26T18:45:36.267 INFO:teuthology.task.internal:Not uploading archives. 2024-03-26T18:45:36.268 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-26T18:45:36.281 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-26T18:45:36.281 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T18:45:36.284 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T18:45:36.286 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T18:45:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: 265305 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 18:45 /home/ubuntu/cephtest 2024-03-26T18:45:36.298 INFO:teuthology.orchestra.run.smithi115.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 18:45 /home/ubuntu/cephtest 2024-03-26T18:45:36.307 INFO:teuthology.orchestra.run.smithi171.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 18:45 /home/ubuntu/cephtest 2024-03-26T18:45:36.308 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-26T18:45:36.370 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-26T18:45:36.398 INFO:teuthology.nuke:Checking targets against current locks 2024-03-26T18:45:36.429 DEBUG:teuthology.nuke:shortname: smithi049 2024-03-26T18:45:36.430 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T18:45:36.452 DEBUG:teuthology.nuke:shortname: smithi115 2024-03-26T18:45:36.452 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T18:45:36.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:33:06.801470', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKSwWzaVsQgro5CHYdbCGUCyWtTXpzEb4NBu7ezmRUIidELmbvYNSdez4ZexFL60G2cSIOOcpRf8NeeQ9TXvSw='} 2024-03-26T18:45:36.475 DEBUG:teuthology.nuke:shortname: smithi171 2024-03-26T18:45:36.476 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T18:45:36.479 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:33:06.799714', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6KAuB0jh54iZe4a5iZSyUd3xUe3stTD7sw9okSnwMcaeWShUX0nxsPHLCM+ZLhRp3JcOQj8I6QzwvxKMlbI2U='} 2024-03-26T18:45:36.502 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623762', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:33:06.802503', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLr9QCKMWUvmLDACiovXBf53HiWiFU500HXopFgmLU+nelvqWgui3p7LPK/xfs1uefYDg0A95ebaoqhCP7SyEg='} 2024-03-26T18:45:36.510 INFO:teuthology.orchestra.console.smithi049:Power off 2024-03-26T18:45:36.511 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T18:45:36.533 DEBUG:teuthology.orchestra.console.smithi049:power off output: Chassis Power Control: Down/Off 2024-03-26T18:45:36.533 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:45:36.550 INFO:teuthology.orchestra.console.smithi115:Power off 2024-03-26T18:45:36.551 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T18:45:36.565 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-03-26T18:45:36.576 DEBUG:teuthology.orchestra.console.smithi115:power off output: Chassis Power Control: Down/Off 2024-03-26T18:45:36.576 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:45:36.590 INFO:teuthology.orchestra.console.smithi171:Power off 2024-03-26T18:45:36.590 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T18:45:36.600 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-03-26T18:45:36.612 DEBUG:teuthology.orchestra.console.smithi171:power off output: Chassis Power Control: Down/Off 2024-03-26T18:45:36.613 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:45:36.640 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-03-26T18:45:40.565 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:45:40.601 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:45:40.641 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:45:40.676 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2024-03-26T18:45:40.711 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-03-26T18:45:40.752 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-03-26T18:45:44.681 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:45:44.711 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:45:44.753 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:45:44.792 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is off 2024-03-26T18:45:44.821 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2024-03-26T18:45:44.862 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is off 2024-03-26T18:45:44.893 INFO:teuthology.orchestra.console.smithi049:Power off completed 2024-03-26T18:45:44.963 INFO:teuthology.orchestra.console.smithi171:Power off completed 2024-03-26T18:45:48.826 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:45:48.935 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off 2024-03-26T18:45:49.036 INFO:teuthology.orchestra.console.smithi115:Power off completed 2024-03-26T18:45:49.141 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/postgres}} duration: 3712.572109222412 failure_reason: '"2024-03-26T18:01:47.476275+0000 mon.a (mon.0) 1147 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_lflores@teuthology success: false 2024-03-26T18:45:49.142 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T18:45:49.235 INFO:teuthology.run:FAIL